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ABSTRACT 


In  1979,  a  F*3  aircrew  training  program,  designed  using  the 
Instructional  Systems  Development  process,  was  implemented  in  the  two  P-3 
Fleet  Replacement  Squadrons.  One  squadron  adopted  workbooks 
recommended  by  ISD,  one  continued  with  lectures.  This  thesis  compares 
the  cost  differences  arising  from  two  different  methods  of  instruction  now 
in  use  in  P-3  training  programs.  A  conceptual  model  is  developed  to 
compare  the  opportunity  costs  associated  with  the  selection  of  one  training 
method  over  another.  The  results  indicate  that  the  lecture  method  of 
instruction  is  least  costly. 
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I.  IN'kODDCTION 


This  thesis  will  compare  the  cost  differences  arising  from  two 
different  methods  of  instruction,  a  technology  based  method  that  uses 
workbooks  as  the  primary  means  of  information  transfer,  and  a 
conventional  instruction  method  that  uses  lectures.  The  purpose  of  the 
comparison  is  to  determine  which  method  has  higher  life-cycle  costs. 
Because  the  selection  of  one  training  method  over  another  has  opportunity 
costs  associated  with  it,  many  of  those  costs  will  also  be  discussed  in  an 
attempt  to  provide  the  decision  maker  with  a  complete  description  of 
pecuniary  and  non-pecuniary  costs  which  are  relevant  to  training  method 
selection. 

Efficient  and  effective  training  is  of  critical  concern  to  the  Navy 
because  of  the  thousands  of  men  and  women  that  must  be  trained 
annually.  Navy  training  systems  typically  cost  more  than  civilian 
programs  because  Navy  students  are  paid  while  attending  training. 
Additionally,  the  consequences  of  poor  training  are  high  in  the  Navy 
because  of  the  ramifications  of  poor  performance  while  serving  in  one  of 
the  many  billet's  with  critical  responsibilities.  [Ref.  1:  pp.  1,3] 

The  high  costs  of  military  training  programs  drive  the  need  for  the 
military  to  weigh  carefully  alternative  methods  of  instruction  so  the  lowest 
cost  alternative  that  accomplishes  the  desired  level  of  training 
effectiveness  can  be  selected.  Kearsley,  in  his  book  Training  and 
Technology,  suggests  that  technology  based  training  programs  are  almost 
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always  cheaper  in  the  long  run  than  a  conventional  method  of  instruction 
[Ref.  2:  p.  153].  Though  this  perception  may  sometimes  be  true,  an 
appropriate  training  method  should  be  sdlected  only  after  a  cost- 
effectiveness  study  closely  examines  each  method.  Unfortunately,  because 
of  the  unique  requirements  of  different  training  programs,  detailed  cost 
data  for  ready  comparison  is  lacking  [Ref.  3:  p.  22]. 

A.  BACKGROUND 

1.  Military  Training  and  Readiness 

The  military  has  training  programs  because  its  readiness  is 
critical  to  its  mission.  Military  readiness  is  dependent  on  personnel 
readiness  which  in  turn  is  affected  by  training,  job  experience,  leadership, 
motivation,  selection  and  other  factors.  Training  design  technologies  have 
been  applied  to  military  training  programs  in  an  effort  to  improve 
readiness.  [Ref.  1:  p.  1]  The  technology  commonly  used  in  the  Navy  for 
designing  training  programs  is  a  method  called  Instructional  Systems 
Development. 

2.  Inatrxictional  Systems  Dev^pment  in  the  Navy 
Instructional  Systems  Development  (ISD)  was  mandated  for 

training  program  development  by  the  Chief  of  Naval  Operations  in  the  mid 
1970's.  The  ISD  method  is  a  performance  based  system  that  uses  a  step 
by  step  procedure  to  design  and  develop  training  programs  by  first 
identifying  the  required  on-the-job  performance.  Following  performance 
determination,  job  tasks  are  determined  from  which  traiiung  objectives  can 
be  developed.  In  later  steps  the  training  objectives  are  sequenced  in  a 
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logical  order,  the  delivery  media  are  selected  and  developed,  and  the 
program  is  implemented.  The  final  .stage  of  the  ISD  process  is  a  control 
phase  based  on  continuing  internal  and  external  evaluation  to  revise  the 
training. 

3.  ISD  and  the  P-3  Aircraft  Training  Programs 

The  Navy  contracted  with  a  civilian  firm  in  1975  to  revise  the 
aircrew  training  program  for  the  lauodbased  P-3  anti-submarine  warfaure 
aircraft  using  ISD  methods  [Ref.  4:  p.  v].  Revision  was  felt  to  be  essential 
because  of  numerous  shortcomings  is  the  training  program  that  were 
identified  in  an  audit  conducted  by  the  P-3  training  squadron  and  the 
Naval  Training  Device  Center  [Ref.  5:  pp.  13,14].  The  Chief  of  Naval 
Operations  directed  that  the  new  training  program  follow  a  format  similar 
to  the  one  used  for  the  S-3A  aircraft,  which  is  a  carrier  based  aircraft 
with  a  mission  similar  to  the  P-3  [Ref.  4:  p.  1].  The  new  P-3  aircrew 
training  program  was  intended  to  replace  existing  training  at  the  two  P-3 
Fleet  Replacement  Squadrons  (FRS),  Patrol  Squadron  Thirty-One  (VP-31) 
located  at  Naval  Air  Station  (NAS)  Moffett  Field,  California,  and  Patrol 
Squadron  Thirty  (VP-30)  located  at  NAS  Jacksonville,  Florida. 

4.  Media  Selection  for  the  P-3  FRS 

During  the  ISD  process,  the  instructional  media  for  the  training 
program  were  selected.  The  media  selected  were  workbooks,  slide-tapes, 
videotapes,  and  lectures/seminars.  [Ref.  4:  pp.  9,bl-b8]  The  workbooks 
were  structured  in  a  manner  similar  to  programmed  instruction,  which 
allows  students  to  proceed  through  the  units  at  their  own  pace.  However, 
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dub  to  the  need  for  group  instruction  in  simulators,  aircraft  labs,  and 
flights,  the  workbook  ctirdculum  remained  group-paced. 

As  time  went  on,  using  a  static  medium  (workbooks)  to  present 
frequently  changing  information  led  to  a  high  number  of  content  errors 
in  many  workbooks.  The  production  process  used  to  update  the 
workbooks  could  not  make  workbook  changes  as  fast  as  they  were 
occurring.  Because  of  dissatisfaction  in  the  workbook  method,  VP-30,  the 
east  coast  training  squadron,  changed  its  training  program  to  a  lecture 
based  system.  VP-31,  the  program  manager  for  the  curriculum,  continued 
to  use  the  workbooks  as  the  primary  method  of  instruction  but  added 
lectures  as  necessary  to  make  up  for  the  workbook  shortcomings. 

B.  RESEARCH  QUESTIONS 

This  study  compares  the  costs  of  the  two  methods  of  instruction  used 
in  the  United  States  Navy's  P-3  aircraft  Fleet  Replacement  Squadron's. 
One  method  uses  workbooks,  which  require  a  complex  production  .system 
to  maintain  and  update  materials  but  has  the  potential  for  reducing 
mempower  requirements.  The  other  method  is  a  conventional  training 
system  that  uses  lectures.  Since  the  fleet  squadrons  of  the  east  and  west 
coast  continue  to  operate  at  high  levels  of  readiness  even  though  the 
graduates  that  join  the  squadrons  are  taught  with  different  instructional 
media,  this  study  assumes  that  both  training  commands  are  meeting  the 
minimum  required  level  of  effectiveness,  and  that  the  effectiveness  of  both 
training  commands  is  the  same.  The  costs  used  in  this  study  include  only 
those  that  are  related  to  the  classroom  training  methods.  Other  costs 
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such  as  aircraft,  fuel,  spare  parts,  aircraft  maintenance,  simulators,  and 
squadron  administration  are  not  examined. 

This  study  gives  a  detailed  description  of  the  data  reqviired  to 
measure  the  costs  of  a  training  method.  Two  different  methods  of 
instruction  used  in  the  Navy  are  compared  to  determine  which  method  is 
more  costly  over  the  life-cycle  of  the  course.  The  opportunity  costs  of 
various  attributes  associated  with  the  training  media  are  also  compared. 

C.  ORGANIZATION  OF  THE  STODY 

Chapter  II  presents  a  brief  historical  background  of  Instructional 
Systems  Development  and  its  subsequent  use  to  develop  the  P-3  training 
program,  and  discusses  several  problems  arising  from  the  media  chosen 
during  the  Instructional  Systems  Development  process.  Chapter  III  is  a 
literature  review  of  works  that  are  pertinent  to  this  study.  In  chapter 
IV,  the  cost  model  used  to  compare  training  alternatives  is  developed. 
Chapter  V  presents  the  data  and  results  from  using  the  cost  model.  A 
summary  and  conclusions  are  presented  in  chapter  VI. 
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n.  HISTORICAL  DEVELOPMENT 


This  chapter  presents  a  brief  historical  account  of  the  events  leading 
to  the  training  programs  now  in  use  at  VP-31  and  VP-30,  and  a 
description  of  the  Instructional  Systems  Development  process  upon  which 
the  training  programs  were  based.  The  historical  development  is  useful 
because  it  illustrates  how  two  programs,  though  baaed  on  the  same 
training  development  methodology,  can  differ  significantly  in  their  teaching 
methods.  It  also  serves  as  a  reminder  of  some  of  the  problems  that  can 
arise  when  adopting  a  new  training  methodology.  Since  the  costs  of  the 
training  methods  of  the  two  squadrons  will  be  compared  in  this  study, 
this  discussion  provides  a  useftil  framework  for  underst3mding  how  those 
cost  differences  developed. 

A.  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT  IN  THE  MILITARY 

Because  of  the  importance  of  training  in  the  military,  the  military 
services  have  been  active  participants  in  the  development  and  use  of  new 
training  technologies.  A  curriculum  development  technique  called 
Instructional  Systems  Development  (ISD)  evolved  from  the  belief  that  the 
systems  analysis  approach,  developed  in  the  1940's,  could  be  used  to 
simplify  training  program  development.  This  technique  has  been  the 
required  standard  for  training  program  design  in  the  Navy  since  the  mid 
1970's.  [Ref.  1:  p.  2] 
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The  ISD  approach  gained  popularity  in  all  military  service  branches 
during  the  1960s,  and  has  been  the  most  pervasive  and  influential  of  the 
training  development  methods  [Ref.  2:  p.  83],  Without  a  formal  approach 
to  training  development,  media  selection  and  course  development  is  more 
of  an  art.  The  ISD  method  turns  course  development  into  a  sequenced 
procedure  that  identifies  training  reqxiirements  and  uses  set  procedures 
to  ensure  that  the  training  program  meets  the  training  objectives.  Two 
major  goals  of  ISD  are: 

to  make  training  (1)  job  relevant  (meaning  it  would  ultimately  prepare 
the  trainee  for  his  or  her  function[s3  in  Navy  readiness)  and  (2) 
cost-effective  and  efficient  (meaning  it  woxild  use  the  most  efficient 
training  methods  to  do  the  training).  [Ref.  1:  p.  4] 

Although  ISD  is  often  mistakexily  confused  with  teaching  methods  or 

techniques  it  is  much  broader  than  that.  Teaching  methods  are  selected 

during  one  phase  of  the  ISD  process,  and  a  training  program  developed 

with  ISD  could  include  several  teaching  methods  such  as  lectures,  self- 

paced  study,  or  computer  based  instruction.  [Ref.  1:  p.  4] 

B.  THE  PROCESS  OF  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT 
1.  The  Instructional  Systems  Dev^pinent  Method 

Kearsley,  in  his  book  Training  and  Technology,  describes  ISD  as 
"a  set  of  procedures  for  developing  instructional  programs  in  a  consistent 
and  reliable  manner."  [Ref.  2:  p.  83]  Developing  training  requirements 
based  on  explicit  job  performance  criteria  is  the  cornerstone  of  the  ISD 
process.  The  content  of  a  training  program  is  determined  only  after  an 
in-depth  study  of  the  job  tasks  of  the  position  to  be  filled.  [Ref.  2:  p.  83] 
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Differences  between  ISD  and  the  traditional  method  of  training  program 
development  are  summarized  in  Table  2.1. 


TABLE  2.1.  COMPARISON  OF  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT 
AND  TRADITIONAL  APPROACHES  TO  INSTRUCTION 


ISD 

Approach 

Content  is  based  upon 
performance  requirements. 

Instructional  objectives  are 
explicitly  stated. 

Content  is  developed  by 
multidisciplinary  team. 

Evaluation  is  part  of  development 
process. 

Media,  strategies,  and  sequencing 
are  analytically  derived. 

Materials  take  into  account 
students  capabilities  and 
differences. 

Development  process  is 
consistent  and  reliable  across 
individuals. 

[Ref.  2:  p.  84] 


Traditional 

Approach 

Content  is  based  upon  topics  to 
be  covered. 

Instructional  objectives  are 
implicit. 

Content  is  primarily  developed 
by  instructor. 

Evaluation  occurs  after  training 
is  implemented. 

Media,  strategies,  and  sequencing 
are  intuitively  selected. 

Materials  are  designed  for  ideal 
or  average  student. 


Development  process  is 
idiosyncratic  and  varies  with 
instructor. 


Training  program  development  in  the  Navy  using  the  ISD  method 
was  set  forth  from  a  series  of  documents  published  by  the  Navy  as 
Interservice  Procedures  for  Instructional  System  Development  (NAVEDTRA 
106A).  Figure  2.1  depicts  the  ISD  process  as  described  in  NAVEDTRA  106A. 
In  this  model,  five  phases  are  used  to  develop  a  training  program.  The 
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ANALYZE  DESIGN  DEVELOP  IMPLEMENT  CONTROL 


Adapted  from  Ref.  5:  p.  26. 

Figure  2.1.  Five  Phases  of  the  ISD  Model 
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first  and  most  important  phase  is  the  analysis  phase.  This  phase 
determine?  if  a  need  exists  for  the  training  in  the  first  place,  and,  if  the 
need  exists,  what  will  be  taught.  Job  analysis,  during  this  phase, 
identifies  the  criteria  of  effective  job  performance.  These  criteria  become 
the  basis  for  the  material  to  be  taught.  [Ref.  6:  p,  1.1] 

The  course  of  instruction  is  designed  during  the  second  phase 
using  the  material  from  the  analysis  phase.  The  objectives  of  the  course 
are  identified,  and  the  sequence  of  instruction  over  which  the  learning 
objectives  will  be  presented  is  determined.  Criterion  referenced  tests  that 
measure  the  degree  to  which  the  students  have  attained  the  learning 
objectives  are  written.  These  tests  are  designed  to  determine  the 
student's  performance  relative  to  the  external  job  requirements.  [Ref.  6: 
PP.  2.1,2.31 

During  phase  three,  the  different  learning  strategies  which  will 
be  used  to  teach  the  students  the  knowledge  required  by  the  performance 
based  objectives  are  specified.  The  knowledge  objectives  are  those  that 
require  the  student  to  recall  information  but  not  apply  it  directly. 
Performance  objectives  require  the  student  to  use  the  information  learned 
to  solve  a  problem  or  operate  equipment. 

Also  in  phase  three,  a  curriculum  outline  is  made  by  the  program 
developers  to  aid  in  planning  the  course,  and  it  subsequently  serves  as 
a  curriculum  approval  document.  Instructional  materials  are  also 
developed.  Writing  the  tests  before  developing  the  instructional  material 
helps  keep  the  program  lean  in  the  sense  of  limiting  the  instruction  to  the 
material  needed  to  develop  student  proficiency  in  the  objectives. 
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Validation  of  the  instructional  materials  is  conducted  through 
internal  review  by  the  designers,  individual  student  trials  using  rough 
drafts  of  the  trairung  materials,  and,  finally,  group  trials  using  rough 
drafts  that  have  been  changed  to  reflect  any  problems  noted  during 
internal  and  individual  validation.  Final  validation  is  conducted  on  a 
regular  class  in  the  normal  training  environment,  and  measures  student 
entry  level  behavior  and  course  completion  performance.  [Ref.  6:  pp. 
3.1,3.70,3.100,3.174-3.186] 

Phase  four  is  the  implementation  phase.  Here,  the  procedures 
used  to  conduct  the  course,  such  as  scheduling,  instructing,  and 
monitoring,  are  specified.  The  final  phase,  phase  five,  is  the  controlling 
phase  of  the  ISD  program.  In  this  phase,  the  training  program  is 
continuously  evaluated  with  respect  to  meeting  the  training  requirements 
using  internal  measures  such  as  student  and  instructor  feedback  reports, 
and  an  in-depth  annual  course  review.  An  external  evaluation  is 
conducted  to  determine  i£  the  graduates  can  do  the  job  for  which  they 
were  trained.  External  evaluation  methods  can  range  from  informal 
discussions  between  the  course  instructors  and  those  who  supervise  che 
course  graduates,  to  surveys  and  structured  interviews.  [Ref.  6:  pp. 
4.1,5.1-5.32] 

2.  ISD  as  Applied  to  the  P-3  Training  Squadrons 

In  the  mid  1970’s,  Courseware,  Incorporated,  was  contracted  to 
redesign,  develop,  implement,  and  evaluate  the  P-3  aircrew  training 
program.  Courseware  worked  with  a  Navy  and  Civil  Service  team  of 
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subject  matter  experts  and  training  specialists  to  develop  the  new  program 
using  ISD  methods.  [Ref.  4:  pp.  1,2] 

To  select  the  instructional  media  for  the  P-3  training  program, 
a  media  selection  algorithm  adopted  from  the  S-3A  training  program  was 
used.  To  use  the  algorithm,  answers  were  obtained  from  a  seri.es  of  five 
questions  for  each  of  the  objectives  being  taught.  The  questions 
identified  the  expected  strident  behavior  during  testing  (such  as  recall, 
recognition,  or  rule  using),  the  level  of  content  being  taught  (such  as 
familiarization  or  memorization),  the  display  requirements  of  the  material, 
and  the  extent  of  the  memorization  requirements.  The  algorithm  output 
provided  a  priority  listing  of  the  optimal  media  for  delivery  of  the 
instructional  materials.  In  many  instances,  computer  based  instruction  was 
the  first  choice.  However,  because  of  cost  constraints,  computer  based 
instruction  was  rejected  and  the  instructional  media  selected  were 
wor  jooks,  slide-tapes,  videotapes,  and  seminars.  [Ref.  4:  pp.  9,bl-b83 

The  P-3  Instructional  System  Management  Plan,  which  was  the 
implementation  document  for  the  newly  developed  training  program,  shows 
that  workbooks  were  the  principal  media  for  information  transfer,  and  that 
lectures  were  not  developed  under  the  original  ISD  plan.  Seminars,  not 
to  be  confused  with  lectures  in  vrhich  an  instructor  teaches  new  material, 
were  included  in  the  training  plan.  Seminars  were  originally  conceived  as 
informal  and  impromptu  get-togethers  to  provide  an  opportunity  for  the 
instructor  to  present  updated  information  not  yet  included  in  the 
workbooks,  and  to  encourage  students  to  discuss  areas  from  the 
workbooks  in  which  they  were  having  difficulty.  [Ref.  7:  pp.  5,6] 
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C.  MEDIA  DEVELOPMENT 

Workbook  development  for  the  P-3  training  squadrons  required  a 
massive  effort  by  the  training  program  development  team.  Providing 
lessons  in  the  form  of  workbooks  for  all  P-3  aircrew  positions  meant 
creating  roughly  850  different  lessons  averaging  35  pages  per  lesson  [Ref. 
7:  p.  k-5].  Using  workbooks,  also  meant  that  a  major  production  system 
had  to  be  established  to  support  initial  development  and  the  later 
revisions  resulting  from  changes  to  the  aircraft  and  operating  procedures. 
The  production  system  included  an  educational  specialist,  editors, 
secretaries,  a  graphics  shop,  a  print  shop,  and  a  library. 

Once  the  materials  were  developed,  the  validation  phase  should  have 
been  conducted.  Because  of  delays  in  workbook  production  and  the  lack 
of  suitable  test  subjects,  only  about  five  per  cent  of  the  instructional 
material  was  validated  [Ref.  4:  p.  15].  Even  though  procedural  changes 
were  made  based  on  the  validation  of  this  small  sample  of  materials,  doubt 
was  cast  on  the  remaining  materials.  Had  all  of  the  material  been  validated 
perhaps  some  of  the  future  problems  encountered  with  the  workbooks 
could  have  been  avoided. 

With  the  same  training  program  and  media  to  be  used  for  the  east 
and  west  coast  P-3  training  squadrons,  one  squadron,  (VP-31),  was 
assigned  model  manager  responsibilities  for  the  training  program.  As 
model  manager,  VP-31  was  responsible  for  curriculum  oversight,  which 
included  producing  and  updating  the  training  materials  for  the  two 
training  squadrons.  The  production  system  was  set  up  in  hangar  one  at 
NAS  Moffett  Field,  California.  Workbook  production  incorporated  the 


13 


resources  of  two  commands,  VP-31  and  Fleet  Aviation  Specialized 
Operational  Training  Group,  Pacific  Fleet.  VP-31  formed  an  ISD  department 
that  was  responsible  for  "ensure[ing  that]  courseware  materials  (including 
testing  instruments)  for  all  syllabi  are  developed  and  maintained  using  the 
most  current  content  information  available."  [Ref.  7:  p.  22]  The  Specialized 
Training  Group's  responsibility  was  to  provide  secretarial  and  graphics 
support  for  ISD  materials,  to  print  ISD  materials  as  required  for  both  Fleet 
Replacement  Squadron's,  and  to  issue  ISD  materials  to  students  attending 
the  training  at  VP-31  [Ref.  7:  p.  25]. 

D.  EVOLUTION  OF  THE  P-3  TRAINING  PROGRAMS 

The  training  program  developed  using  the  ISD  process  was 
implemented  at  VP-31  in  January  of  1979,  and  at  VP-30  in  January  of  1980 
[Ref.  5:  p.  10].  By  1984,  both  training  commands  had  added  substantial 
lecture  time  because  of  problems  with  the  workbooks.  Numerous  errors  in 
the  workbooks  fostered  frustration  and  a  lack  of  confidence  in  the 
workbook  system.  These  errors  were  the  result  of  a  production  system 
originally  set  up  to  handle  a  much  smaller  workbook  revision  load.  The 
inadequate  staff  was  quickly  overwhelmed  by  the  changes  needed  because 
of  aircraft  modifications,  changes  in  operating  procedures,  or  changes  in 
the  tactical  employment  of  the  aircraft.  The  volume  of  changes  led  to 
lengthy  backlogs  in  workbook  revisions — some  revisions  required  over  a 
year  to  move  through  the  system.  In  an  instructional  systems  status 
letter,  the  Commanding  Officer,  VP-31  stated  "...  by  early  1983  the 
quality  of  the  information  contained  in  the  system  had  slowly  deteriorated 
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to  a  point  that  the  courseware  no  longer  reflected  current  operating 
procedures."  [Ref.  8:  pp.  1-4]  The  lectures  were  added  to  the  training 
program  to  compensate  for  the  workbook  shortcomings  because  they  could 
easily  incorporate  current  information.  An  added  bonus  was  later  noted 
by  a  VP-31  commanding  officer  who  observed  that  lectures  and  workbooks, 
along  with  other  media  in  the  curriculum,  lead  to  a  richer  mix  of  media 
and  a  greater  opportunity  for  the  students  to  grasp  the  objectives. 

Presently,  VP-31  is  predominantly  workbook  oriented,  i.e.,  workbooks 
remain  the  primary  media  for  information  transfer.  A  large  number  of 
lectures  have  been  added,  however,  so  that  the  present  ratio  of  workbooks 
to  lectures  is  one  to  one,  measured  over  all  aircrew  tracks.  VP-30  has 
changed  its  curriculum  so  that  it  is  a  lecture  oriented  training  program. 
Many  of  the  workbooks  are  still  used  at  VP-30,  but  only  for  supplemental 
reading  assignments. 

As  part  of  the  evolutionary  process  within  the  training  command, 
subject  matter  experts  are  changing  the  workbooks  so  that  material 
subject  to  frequent  change  is  deleted.  With  this  material  deleted,  the 
student  is  referred  to  appropriate  source  publications  where  the  material 
is  more  frequently  updf.ted.  This  new  method  should  reduce  the  revision 
requirements,  shorten  the  revision  backlog,  and  reduce  the  student  and 
staff  frustration  resulting  from  out-of-date  materials. 
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in.  LITERATURE  REVIEW 


A.  COST  ANALYSIS  METHODS 
1.  Cost-Benefit  Analysis 

Over  the  past  20  years  there  has  been  a  dramatic  increase  in 
the  analytic  studies  that  are  conducted  to  support  public  policy  decisions. 
Some  of  the  generic  names  applied  to  such  emalyses  are  environmental 
impact  statements,  cost-benefit  analysis,  risk -analysis,  and  decision 
analysis.  These  analyses,  supposedly  explicit  evaluations  of  risks  and 
benefits,  theoretically  provide  the  tools  for  "regulatory  openness."  Clear 
evaluations  of  policies  required  that  the  assumptions  and  numbers  used  to 
derive  the  estimates  also  be  made  available.  Unfortunately,  as  emphasized 
by  Firschhoff  in  The  Art  of  Cost-Benefit  Analysis,  "these  analytic 
techniques  have  both  inherent  limitations  and  potential  for  misuse."  [Ref. 
9:  p.  1.1]  For  analyses  to  be  useful,  the  interested  observers  should  be 
aware  of  the  techniques  that  were  used  and  know  when  the  conclusions 
should  be  heeded.  [Ref.  9:  pp.  1.1, 1.2] 

The  basic  premise  of  a  cost-benefit  analysis  is  that  benefits  and 
costs  should  be  measured  for  the  proposed  action,  and  that  a  new 
technology  or  policy  shoxild  be  adopted  only  if  the  anticipated  benefits 
outweigh  the  anticipated  costs.  The  major  challenge  of  a  cost-benefit 
study  lies  in  the  correct  and  complete  enumeration  of  all  costs  and 
benefits.  [Ref.  9:  p.  2.1] 
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2.  Cost-Effectiveness  l^alysis 

Because  the  quality  of  graduates  from  Navy  training  programs 
affects  the  Navy's  readiness,  training  program  effectiveness  is  a  major 
concern.  Most  of  the  studies  conducted  to  evaluate  or  compare  the 
effectiveness  of  military  training  programs  have  used  a  cost-effectiveness 
approach.  The  studies  are  often  undertaken  to  determine  program  cost  or 
program  effectiveness  and  not  to  determine  if  the  benefits  outweigh  the 
costs.  Often,  the  benefits  of  the  military  training  program(s)  being 
measured  or  compared  are  already  assumed  to  outweigh  the  costs. 

Cost-effectiveness  analysis  provides  a  means  of  assessing  decision 
alternatives  among  training  programs.  When  more  than  one  means  of 
achieving  an  objective  is  available,  cost-effectiveness  analysis  theoretically 
provides  a  means  of  identifying  the  least  cost  alternative.  Orlansky 
suggests  addressing  the  following  three  questions  in  a  cost-effectiveness 
study  [Ref.  3:  p.  6]: 

1.  Does  the  new  technology,  device,  or  method  of  instruction  actually 
improve  the  effectiveness  of  training  and,  if  so,  by  how  much? 

2.  How  much  does  the  new  technology,  device,  or  method  of 
instruction  cost  (preferably  on  a  life-cycle  basis)  compared  to 
present  practice? 

3.  If  the  effectiveness  of  training  is  actually  increased,  is  the  amount 
of  improvement  worth  its  cost? 

These  questions  suggest  several  possible  combinations  of  cost  and 
effectiveness.  The  possible  combinations,  along  with  the  decision  each 
combination  supports,  is  shown  in  Figure  3.1.  If  a  training  alternative  is 
judged  to  have  greater  effectiveness  at  less  cost  or  the  same  effectiveness 
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at  less  cost  it  wovild  appear  to  be  a  preferable  choice.  If  the  alternative 
has  less  effectiveness  at  the  same  or  greater  cost,  or  the  same 
effectiveness  at  higher  cost,  rejecting  the  alternative  would  be  the 
preferred  choice.  The  choice  is  less  clear  cut  when  cost  and  effectiveness 
are  both  determined  to  be  less,  the  same,  or  more  than  the  other 
alternatives  being  compared.  If  cost  and  effectiveness  are  both  higher, 
it  must  be  determined  whether  the  higher  effectiveness  justifies  the 
higher  cost.  If  the  effectiveness  is  lower  but  cost  is  also  lower  it  then 
becomes  a  question  of  whether  the  lower  effectiveness  is  acceptable.  If 
no  difference  in  cost  and  effectiveness  exists  among  alternatives,  then 
final  selection  will  depend  on  other  important  features  such  as  the 
flexibility  of  the  method  under  different  student  loading  or  instructor 
manning  scenarios.  [Ref.  3:  p.  6] 
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Adapted  from  Ref.  3:  p.  7. 

Figure  3.1.  Decision  Matrix  for  Cost  and  Effectiveness 


The  questions  about  costs  and  effectiveness  presented  by 
Orlansky  imply  that  other  features  such  as  flexibility  to  operate  under 
different  student  loading,  do  not  have  a  cost.  However,  close  observation 
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of  a  training  program  may  reveal  ways  to  measure  or  estimate  the  costs 
of  "other  features"  that  are  often  not  included  in  a  cost-effectiveness 
evaluation.  Many  features  that  are  often  overlooked  by  the  analyst  are 
features  of  the  training  program  that  have  an  opportunity  cost  associated 
with  them. 

3.  Opportunity  Cost  of  an  lUtemative 

When  making  a  choice  between  alternatives,  there  is  an 
opportunity  cost  involved  in  that  choice.  To  have  something,  other  things 
must  have  been  given  up,  or  an  opportunity  to  acquire  an  alternative 
thing  must  have  been  forgone.  The  forgone  alternative  is  called  the 
opportunity  cost  of  that  choice.  Resources  used  on  one  project,  for 
example,  are  no  longer  available  for  other  uses.  When  operating  with 
limited  resources,  a  decision  to  proceed  with  one  project  has  an  implied 
decision  not  tc  undertake  other  projects.  [Ref.  10;  p.  110]  This  means 
that  if  all  resources  are  used  on  the  operational  front,  nothing  remains  for 
training,  research,  development,  and  logistics.  At  a  less  aggregate  level, 
the  choice  to  buy  one  more  aircraft  may  mean  fewer  flight  hours  will  be 
allocated  to  operate  the  aircraft.  It  is  therefore  in  the  opportunity  cost 
where  the  real  cost  of  the  purchase  is  found.  Using  opportunity  costs 
rather  than  strict  accounting  costs  can  help  the  decision  makers  determine 
the  true  costs  of  the  alternatives  during  the  decision  process. 

B.  COST-EFFECTIVENESS  STUDIES  COMPARING  MILITARY  TRAINING  MEDIA 

The  common  definition  of  effectiveness,  as  used  in  military  cost- 
effectiveness  studies,  is  described  by  Hall,  Rankin  and  Aagard  as  "  .  .  . 
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(measures  of)  the  degree  to  which  a  training  course  or  system  achieves 
the  goals  established  for  it.”  To  measure  effectiveness,  the  training  goals 
must  be  converted  into  behavioral  terms  that  can  be  measured,  e.c;- 
performance.  [Ref.  11:  p.  12] 

Many  studies  that  have  compared  the  effectiveness  of  different 
instructional  media  used  in  military  training  commands  have  concluded  that 
no  medium  ranks  significantly  higher  than  the  others  in  effectiveness. 
This  result  could  be  because  most  of  the  military's  training  consists  of 
mixed  media  programs.  Although  one  medium,  such  as  lectures  or 
workbooks,  may  be  used  more  than  the  others,  additional  media  or 
instructor  attention  may  compensate  for  its  shortcomings. 

A  comparison  between  individual  and  conventional  instruction  was 
made  by  Freda  and  Hall  [Ref.  13]  in  an  attempt  to  determine  the 
differences  in  effectiveness  and  efficiency.  Data  for  the  study  were 
obtained  from  19  military  training  programs  and  over  5000  students.  Their 
methods  for  the  comparison  involved  statistical  regression  using  data 
obtained  on  five  predictor  and  four  criterion  variables.  The  five  predictor 
variables  were:  the  method  of  instruction,  (conventional  or  individualized), 
the  trednees'  ability  level,  (based  on  armed  forces  qualification  test 
scores),  the  type  of  training  task,  (fact,  category,  procedure,  rule,  or 
principle),  gender  of  the  graduate,  and  the  location  of  the  school.  The 
four  criterion  variables  chosen  were  divided  into  effectiveness  measures 
and  efficiency  measures.  The  effectiveness  measures  were  end-of-course 
grades  and  training  adequacy  ratings  (performance  survey  from  graduate's 
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first  level  supervisor).  The  efficiency  measures  were  cost  of  the  course 
and  time  to  complete  the  course. 

Freda  and  Hall  concluded  that  individualized  and  conventional 
instruction  were  equally  effective  ways  of  preparing  sailors  for  their  jobs. 
Within  the  two  categories,  students  of  individualized  instruction  with 
greater  ability  (mental  categories  1  and  2)  had  higher  course  grades  and 
finished  in  a  shorter  time  than  lower  mental  category  students,  but  in 
conventional  instruction,  no  differences  were  seen  among  mental  categories. 
[Ref.  12:  p.  69] 

Freda  and  Hall  were  not  able  to  make  a  definitive  judgement  that  one 
training  method  was  more  expensive  than  the  other.  For  the  cou..-es 
compared,  the  cost  of  individual  instruction  was  higher  than  that  of 
conventional  instruction,  but  the  courses  compared  were  of  different 
lengths  and  were  not  similar  in  type  of  material.  To  determine  costs,  the 
researchers  used  the  "per  capita  course  costing  data  base"  from  the  Chief 
of  Naval  Education  and  Training.  This  data  base  does  not  contain 
research  and  development  costs  and  also  lacks  many  of  the  investment 
costs,  so  the  comparison  is  primarily  based  on  differences  in  student  and 
instructor  salaries.  [Ref.  12:  pp.  69,D.l] 

Braby  and  Evans  conducted  a  study  that  was  a  comparison  of  six 
elements  of  quality  between  individual  and  conventional  instruction  for  37 
courses.  The  quality  of  instruction  in  each  type  of  course  was  determined 
with  a  50  question  survey  administered  to  the  1,090  students,  170 
instructors,  and  54  course  supervisors.  A  structured  interview  was  also 
conducted  at  the  training  command  of  each  of  the  37  courses  with  the 
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person  that  was  "deemed  ’most  knowledgeable'  of  the  course  by  one  of  the 
two  principal  investigators."  [Ref.  13:  pp.  15,17]  The  concluding  remarks 
of  Braby  and  Evans  were: 

Instructional  strategy  (II  vs  Cl)  [individual  instruction  vs 
conventional  instruction]  is  not  the  determining  factor  in  overall 
course  effectiveness  of  current  Navy  courses.  Rather,  effectiveness 
is  mediated  by  the  extent  to  which  good  instructional  practices  are 
used  within  the  courses  and  ultimately  determined  by  the  degree  to 
which  proper  practices  can  be  employed  within  various  instructional 
strategies.  Since  it  is  possible  for  properly  executed  II  and  Cl  to  be 
equally  effective,  a  choice  of  strategy  should  rest  primarily  on  the 
relative  cost  efficiencies  of  the  two  approaches.  [Ref.  13:  p.  33] 

There  are  many  ways  to  approach  a  cost-effectiveness  study  but  it 
is  difficult  to  compare  the  costs  of  policies  or  programs  with  conclusions 
from  analyses  using  different  methods.  In  a  1977  Rand  Corporation  paper 
[Ref.  15]  Marks  and  Massey  examined  several  cost  models  used  in  Air  Force 
programs  and  various  case  studies  of  non-Air  Force  programs.  They 
concluded  that  many  of  the  studies:  were  inadequate  as  a  basis  for 
making  "a  sound  resource  decision,"  were  lacking  in  the  depth  of  analysis 
required  for  cost  elements,  and,  used  "poorly  defined  baselines  against 
which  to  measure  the  cost  impact  of  alternative  cases."  [Ref.  14:  pp. 


5,12,14] 


In  1983,  Knapp  and  Orlansky  developed  a  cost  element  structure  for 
defense  training  with  the  purpose  of  providing  a  standardized  approach 
to  determining  the  costs  of  a  training  program,  regardless  of  size.  The 
cost  element  structure  is  a  listing  of  areas  where  costs  should  be 
measured  and  is  relevant  to  a  diverse  range  of  cost  studies  if  the  analyst 
simply  disregaurds  the  cost  measurement  areas  not  applicable  to  his  study. 
The  format  recommended  in  the  study  requires  determining  program  costs 


22 


by  measuring  costs  of  research  and  development,  investment  costs,  and 
operating  costs.  [Ref.  15:  pp.  1,8-16]  The  format  adds  only  slightly  more 
detail  than  previous  studies  and  doesn't  provide  information  on  how  to 
obtain  the  costs.  The  study  describes  all  costs  that  Knapp  and  Orlansky 
feel  would  have  an  effect  on  the  budget  of  an  organization,  but  a 
weakness  of  the  approach  is  a  resulting  ignorance  of  the  opportunity 
costs  that  are  associated  with  a  specific  training  program. 

Many  studies  have  been  conducted  using  a  strategy  similar  to  that 
of  the  cost  element  structure.  Several  have  compared  different  military 
training  methods.  In  1981,  Corey  compared  the  costs  of  training  at  RM-A 
and  IC-A  school  in  an  attempt  to  determine  whether  individualized 
instruction  or  conventional  instruction  was  cheaper.  As  in  the  cost 
element  approach,  Corey's  approach  was  to  identify  the  resources  reqxrired 
for  the  instruction  and  then  to  estimate  the  cost  of  the  resources.  The 
least  cost  alternative  producing  a  given  output  was  taken  to  be  the  most 
efficient.  For  this  study,  output  was  defined  as  the  number  of  graduates 
trained  to  a  certain  level.  All  methods  of  instruction  were  assumed  to 
have  the  same  level  of  effectiveness.  [Ref.  16:  p.  13] 

Cost  comparisons  were  made  using  estimated  investment  and  operating 
costs  of  the  training  programs.  The  one-time  investment  costs  included 
costs  for  curriculum  development,  specialized  programming,  class  facilities, 
and  equipment.  The  annual  operating  costs  included  costs  of  staff, 
students,  hardware  lease  costs,  facilities,  maintenance,  supplies,  and 
curriculum  maintenance  costs.  The  staff  costs  used  the  Navy  Life-Cycle 
Billet  Cost  Model,  whic.i  estimates  annual  costs  to  the  government  of 
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maintaining  billets  of  various  rates  and  ratings.  These  costs  include  the 
individuals'  pay,  allowances,  and  benefits.  [Ref.  16:  pp.  14,15] 

Using  a  billet  cost  model  that  includes  the  cost  of  past  instructor 
training  as  in  this  study  could  be  misleading.  The  past  training  is  a  sunk 
cost  that  was  expended  to  develop  the  expertise  of  the  instructor  for  his 
operational  squadron  and  is  not  relevant  to  determining  the  present  costs 
of  training  alternatives.  If  the  required  manning  of  the  alternatives  being 
compared  is  identical,  then  the  added  traiiung  cost  will  cause  the  total 
cost  of  an  alternative  to  be  overinflated,  but  will  not  affect  the  outcome 
of  the  decision  among  alternatives.  If  the  manpower  requirements  of  the 
programs  differ,  and  alternatives  are  available,  then  the  BiUet  Cost  Model 
could  influence  the  choice.  Billet  costs  that  include  aircrew  traiiung  would 
tend  to  become  very  high  because  of  the  expense  of  aviation  training. 

The  results  of  Corey's  s'cudy  show  that  individual  instruction  is  less 
costly  than  conventional  instruction  over  a  15  year  program  life.  This 
result  however,  is  sensitive  to  student  loading  and  course  length.  When 
student  loading  was  less  than  440  students  per  year,  conventional 
instruction  was  less  expensive  than  individualized  instruction.  The  higher 
cost  of  conventional  instruction  above  the  440  students  per  year  point  is 
attributed  to  the  longer  period  of  time  (longer  than  individual  instruction) 
required  for  the  student  to  remain  in  the  training  command.  Sensitivity 
analysis  suggested  that  if  the  conventionally  taught  RM-A  course  was 
shortened  to  eight  weeks,  conventionad  instruction  was  less  costly.  It  is 
interesting  to  note  how  the  student  loading  or  course  length  swings  the 
balance  of  which  course  is  more  expensive. 
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A  potential  weakness  of  the  study  is  that  for  the  comparisons  a 
hypothetical,  conventionally  taught  training  program  was  used.  The 
hypothesized  conventional  method  of  instruction  was  estimated  to  require 
20%  more  time  to  complete  than  individualized  instruction.  The  estimate 
was  based  on  previous  studies  that  examined  learning  rates  associated  with 
different  methods  of  instruction.  Another  possible  weakness  is  a  failure 
to  use  measures  of  opportunity  cost  when  comparing  the  alternatives. 

Another  cost  model  to  determine  the  costs  of  potential  changes  in  a 
training  program  was  developed  by  Thode  and  Walker.  Fortunately,  Thode 
and  Walker  were  involved  in  the  ISD  effort  conducted  at  VP-31  in  the  late 
1970's  and  detailed  cost  information  of  the  ISD  effort  and  of  VP-31's 
training  program  are  available.  Their  model  uses  a  structure  similar  to 
the  cost  element  structure  proposed  by  Knapp  and  Orlansky.  The  study 
provides  a  working  example  of  the  model  with  a  comprehensive  analysis  of 
all  major  costs  of  the  P-3  training  program  as  envisioned  by  the  ISD 
process  including  aircraft,  maintenance,  administrative,  instructor,  and 
student  costs.  [Ref.  17:  pp.  1-4] 

Lacking  in  the  Thode  and  Walker  study  is  any  comparison  of 
alternative  teaching  methods,  although  the  model  could  be  used  to  make 
such  a  comparison.  Their  model  improves  on  other  cost  studies  by 
including  the  opportunity  costs  associated  with  updating  the  media,  and 
provides  greater  detail  on  the  origin  of  the  costs.  Because  of  the  way  in 
which  the  cost  data  were  aggregated  in  the  study,  it  is  impossible  to 
determine  many  of  the  costs  that  would  be  useful  for  a  comparison  of 
media  development  and  update  costs. 
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The  work  presented  here  avoids  the  estimations  reqxiired  for  a 
hypothesized  training  program  because  the  training  commands  studied  have 
identical  training  goals  although  they  use  different  training  media  to 
achieve  those  goals.  In  addition,  opportunity  costs  are  included  in  the 
cost  comparison  of  the  training  programs. 
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IV.  METHODOLOGY 


A  conceptual  model  is  useful  for  categorizing  and  evaluating 
opportunity  costs  when  comparing  the  costs  of  alternative  instructional 
media.  The  conceptual  model  developed  in  this  chapter  is  an  adaptation 
of  the  Thode  and  Walker  model.  It  categorizes  costs  as  plant  and 
equipment,  development,  or  operating  costs,  and  extends  the  Thode-Walker 
model  by  identiEying  the  opportunity  costs  associated  with  each 
alternative.  The  model  is  used  to  compare  the  costs  of  the  instructional 
media  of  workbooks  and  lectures  as  used  in  the  classroom  segment  of  the 
P-3  aircrew  training  programs;  as  stated  previously  the  training  costs 
incurred  by  such  things  as  aircraft,  maintenance,  facilities,  and 
administration,  are  assumed  constant.  The  model,  shown  in  Figure  4.1, 
divides  costs  into  three  major  categories,  plant  and  equipment, 
development  costs,  and  operating  costs  for  media  comparison.  Cost 
comparisons  were  done  for  seven  different  training  tracks  within  the  Fleet 
Replacement  Squadron's.  The  tracks  were.  Pilot,  Naval  Flight  Officer, 
Acoustic  Operator,  Non-Acoustic  Operator,  Flight  Engineer,  In-Flight 
Technician,  and  Ordnanceman. 

The  instructional  media  were  determined  in  an  early  phase  of  the 
Instructional  System  Development  (ISD)  training  program  design.  The 
costs  to  develop  the  selected  media  can  vary  considerably  among 
alternatives.  For  materials  development  done  in-house,  cost  estimates 
reqxhre  a  working  knowledge  of  the  system  that  produces  the  materials. 
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Figure  4.1.  Conceptual  Model  to  Determine  Training  Media  Costs 
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For  development  done  under  contract,  contracted  prices  should  be 
available.  If  one  medium  requires  substantially  more  organizational  effort 
and  labor  in  the  production  process,  then  higher  costs  will  result  because 
of  increased  manpower  requirements.  If  the  training  squadron  cannot 
adjust  its  manpower,  then  that  medium  will  result  in  higher  opportunity 
costa  for  the  squadron  because  the  increased  administrative  workload  must 
generally  be  handled  by  the  instructors.  Areas  given  up  by  the  increased 
workload  are  things  such  as  instructor  training  time  which  increases 
instructor  proficiency,  less  attention  applied  to  other  jobs  held  by  the 
instructor,  longer  working  hours,  and,  possible  decreases  in  job 
satisfsiction.  The  training  squadrons  compared  in  this  study  are  manned 
equally,  so  if  one  medium  requires  more  management  effort,  that  increased 
effort  imposes  additional  costs  on  the  squadron.  The  additional  costs  may 
be  poor  lecture  preparation,  degraded  instructor  teaching  performance, 
and  lower  morale  if  there  is  not  enough  time  to  accomplish  all  jobs 
satisfactorily. 

A.  PLANT  AND  EQUIPMENT 

If  workbook  or  lecture  development,  and  later  revisions,  are  done  in- 
house,  the  organization  must  have  or  create  a  structure  that  can  handle 
the  development  and  revision.  Each  medium  has  different  worker  and 
equipment  requirements  necessary  for  development  and  revision.  The 
purchase  price  of  the  equipment  and  its  subsequent  maintenance  cost 
should  be  considered  in  comparing  the  costs  of  different  media. 
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The  equipment  costs  of  the  workbook  method  of  instruction  were 
determined  aicer  discussions  with  the  division  supervisors  of  the 
production  process  and  other  experts  in  the  program.  AU  major  equipment 
needed  to  develop  or  revise  the  workbooks  were  noted  and  prices  were 
assigned  based  on  the  present  purchase  price  of  the  equipment.  The 
major  equipment  costs  were  computers,  printers,  graphics  equipment,  and 
printing  equipment.  Equipment  costs  of  lecture  presentation,  i.e.,  overhead 
projectors  and  slide  projectors,  were  ignored  because  these  items  are 
required  for  both  squadrons  and  are  therefore  not  relevant  to  the 
comparison. 

B.  DEVELOPMENT  COSTS 

The  development  costs  of  the  workbooks  used  by  VP-31  were 
estimated  by  determining  the  amount  of  time  required  for  the  military 
subject  matter  experts  in  each  track  to  write  the  books  and  adding  it  to 
an  estimate  of  the  amount  of  time  required  for  all  others  in  the  production 
process  to  produce  the  books.  Others  in  the  process  were:  aircrew  track 
managers,  educational  specialist,  typists,  editors,  graphics  artists,  and 
printers.  The  time  estimates  for  each  production  worker  were  based  on 
a  survey  of  the  military  track  managers.  The  calculation  of  hourly  wagt.-s 
for  the  subject  matter  experts  multiplied  by  the  production  hours,  summed 
over  all  workbooks,  plus  two  years  of  worker  salaries  for  the  ISD, 
graphics,  and  printing  departments,  were  used  to  determine  the  total 
workbook  development  cost. 
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The  lecture  development  cost  was  estimated  by  multiplying  the  time 
required  for  an  instructor  to  produce  a  lecture  (obtained  from  instructor 
estimates)  by  the  hourly  wage  of  the  instructor  in  the  training  track  for 
which  the  lecture  is  used.  The  costs  for  all  lectures  in  all  tracks  were 
summed  to  arrive  at  the  total  cost  of  lecture  development. 

C.  OPERATING  COSTS 

Media  operating  costs  are  the  costs  associated  with  a  specific  training 
medium  used  to  produce  the  product—the  graduate.  Included  here  are 
the  costs  that  are  a  function  of  the  graduate  production  pr^cess,  the 
costs  of  keeping  training  media  current,  and  costs  of  other  attributes  of 
media  such  as  media  portability  and  the  ability  of  media  to  respond  to 
changes  in  student  loading. 

In  the  aviation  training  program,  the  production  process  is  a 
conglomerate  of  training  activities  that  provides  the  student  with  the 
knowledge,  skills,  and  abilities  to  do  the  job  at  the  completion  of  training. 
The  aviation  training  program  uses  a  media  miy  that  includes  workbooks, 
lectures,  videotapes,  aircraft  simulators,  aircraft  labs,  and  flights.  While 
all  of  these  media  are  critical  to  the  training  program,  the  workbooks  and 
lectures  teach  nearly  all  of  the  theoretical  knowledge  in  the  program. 

Teaching  the  theoretical  knowledge  is  a  time  consuming  process,  and 
one  medium  may  be  able  to  accomplish  it  in  less  time  than  another.  Self- 
paced  instruction  using  workbooks  can  theoretically  reduce  training  time 
because  students  can  proceed  at  their  own  pace.  More  recently,  computer 
based  instruction  is  making  the  same  claim.  In  order  to  take  the 
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possibility  of  reduced  training  time  into  account  the  number  of  hours  the 
students  spend  with  the  workbooks  and  lectures  was  determined  from  the 
training  syllabi  of  each  of  the  seven  training  tracks. 

Measuring  the  difference  in  training  time  alone  may  be  misleading 
because  the  ability  of  the  students  upon  course  completion  may  be 
different.  If  a  student  is  sped  through  an  aircrew  training  program  and 
sent  to  the  fleet  inadequately  prepared,  there  is  an  additional  cost 
imposed  on  the  fleet  squadron  receiving  the  undertrained  graduate.  The 
additional  cost  arises  from  the  additional  time  a  supervisor  must  spend 
with  the  new  person,  the  gas  and  maintenance  incurred  by  remedial  flight 
training,  and  the  reduced  readiness  of  the  squadron  because  of 
inadequately  trained  people. 

There  are  no  indications  that  the  effectiveness  of  the  squadrons 
compared  in  this  study  is  different.  Fleet  squadrons  on  the  east  and  west 
coasts  enjoy  high  levels  of  readiness,  and  neither  training  squadron  is 
receiving  complaints  about  its  training  product.  If  the  training  squadrons 
are  conforming  to  the  training  goals  established  during  the  ISD  process, 
then  they  are,  by  definition,  teaching  only  to  the  specified  level  of 
objectives,  and  the  effectiveness  of  both  squadrons  shovild  be  the  same. 

Along  with  reduced  training  time,  reduction  in  manpower  requirements 
brought  about  by  the  self-paced  nature  of  the  workbooks  were  expected 
by  the  developers.  The  manpower  reduction  was  not  realized  to  the  extent 
originally  expected.  A  "sterile"  workbook  environment  was  felt  to  impede 
learning  because  it  bored  the  students,  and  in  1982,  VP-31  began 
assigning  instructors  to  student  study  areas,  also  called  the  learning 


32 


center,  to  provide  assistance,  answer  questions,  and  get  the  students 
together  for  informal  seminars  in  cases  when  many  students  were  having 
difficulty  with  a  particular  area  of  study.  This  new  policy  increased  the 
opportunity  cost  of  the  workbook  method  of  instruction  because  the 
instructors  now  lost  productive  time  from  their  other  responsibilities;  even 
though  an  instructor  was  on  a  fixed  salary  and  was  paid  the  same  amount 
regardless  of  what  he  was  doing,  the  instructor's  productivity  in  his  other 
assignments  was  decreased  by  added  duties. 

An  estimate  of  the  costs  of  the  production  process  for  workbooks  and 
lectures  uses  the  student  and  instructor  training  time  that  is  associated 
with  the  particular  medium.  It  is  not  correct  to  simply  apply  the  entire 
instructor  and  student  salary  to  these  estimates  because  a  large 
proportion  of  the  training  is  conducted  with  other  media  outside  of  the 
classroom  or  learning  center,  i.e.,  in  ground  trainers,  aircraft  device 
sessions,  and  flights. 

Determining  the  training  time  requirements  for  the  individual  media 
was  done  by  obtaining  the  curriculum  descriptions  of  the  seven  different 
aircrew  training  tracks  of  both  training  squadrons.  The  hours  required 
for  the  workbook  and  lecture  medium  were  determined  from  the  curriculum 
descriptions.  These  hours  were  converted  to  dollars  by  multiplying 
instructor  hourly  earnings  times  the  number  of  hours  required  for  the 
particular  medium,  multiplying  the  student  hourly  earnings  times  the 
number  of  hours  required  i'ir  the  particular  medium,  and  adding  the  two 
together. 
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1.  Media  Currency 

A  continxiing  problem  for  the  P-3  training  program  is  the  fact 
that  the  instructors  must  maintain  training  material  currency.  In  the  last 
ten  years  there  have  been  seven  variants  of  the  P-3  aircraft  to  teach  at 
the  training  command  each  with  its  own  characteristics  (P-3B  mod,  P-3C 
baseline,  P-3C  mod,  P-3C  UI,  P-3C  UII,  P-3C  UII.5,  P-3C  UIII).  As  a  result 
the  content  of  the  training  program  undergoes  frequent  revision  in  an 
effort  to  keep  up  with  equipment  modifications  to  the  aircraft  and  changes 
in  the  aircraft  tactics  that  are  used  in  the  fleet  environment.  With 
frequently  changing  material,  a  desirable  feature  of  any  training  medium 
would  be  the  ability  to  make  changes  at  relatively  low  cost.  Lectures  and 
workbooks  differ  substantially  in  this  respect:  major  workbook  revisions 
may  require  six  months  or  more  to  go  through  the  production  system,  but 
changes  to  lectures  can  usually  be  accomplished  in  several  hours  and  the 
new  material  can  be  presented  at  the  next  lecture. 

Workbooks  developed  under  the  ISD  process  take  up  to  year  to 
revise  because  revisions  to  them  require  the  efforts  of  many  people. 
Subject  matter  experts  must  review  the  old  workbooks  for  content 
currency,  note  areas  that  need  revision,  and  write  new  sections  to  reflect 
the  aircraft  hardware  changes.  Track  managers  who  administer  the 
workbooks  assigned  to  their  tracks  must  note  changes  to  the  aircraft  or 
software  and  initiate  the  change  procedure.  Educational  specieilists  must 
take  the  input  from  the  subject  matter  experts  and  make  changes 
necessary  to  ensure  that  the  workbook  continues  to  accomplish  its  original 
purpose.  Editors,  typists,  and  graphic  artists  work  to  make  the  final 
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copy.  Only  then  can  the  new  material  go  to  press,  which  requires  the 
efforts  of  the  people  in  the  printing  division.  In  contrast,  an  instructor 
updating  his  lecture  may  request  some  graphics  support  for  updating 
transparencies  or  class  handouts,  but  the  normal  delay  is  a  matter  of 
hours.  In  the  case  of  major  revisions,  revising  lectures  may  require 
several  days  at  the  most. 

The  costs  of  maintaining  media  currency  can  be  determined  by 
estimating  the  costs  to  revise  the  media.  Measures  of  revision  costs  are 
not  readily  available  because  they  vary  from  one  type  of  training  to 
another  and  depend  on  how  frequently  the  course  content  changes. 
Estimates  of  these  costs  must  be  obtained  from  the  person  that  makes  the 
changes.  Even  then  the  quoted  number  of  hours  are  usually  estimates  of 
what  is  perceived  as  typical,  and  are  not  based  on  specific  records.  For 
this  study,  however,  estimates  of  the  number  of  revisions  per  year  and 
the  size  of  the  revisions  are  based  on  the  records  kept  for  workbook 
revision.  The  revision  costs  of  workbooks  used  by  VP-31  were  estimated 
in  a  manner  similar  to  development  cost  estimates. 

The  time  estimates  for  each  person  in  the  revision  process  were 
based  on  a  survey  of  the  military  track  managers.  The  hourly  wages  of 
those  involved  in  the  changes  were  multiplied  by  the  time  required  to 
make  the  changes.  Total  costs  were  obtained  by  summing  the  changes  for 
all  workbooks  over  all  aircrew  tracks.  The  wages  of  production  workers 
other  than  instructors  who  were  assigned  to  the  ISD,  graphics,  printing 
departments,  were  included  because  they  exist  to  make  the  revisions.  The 
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annual  operating  budgets  of  these  divisions  were  also  included,  and 
provide  an  estimate  of  the  cost  of  supplies  used  in  the  process. 

The  costs  to  revise  the  lectures  used  by  VP-30  and  VP-31  were 
estimated  by  multiplying  the  time  required  for  an  instructor  to  revise  a 
lecture  by  the  hourly  salary  of  the  instructor  responsible, for  the  lecture. 
The  costs  for  all  lectures  of  all  tracks  were  summed  to  arrive  at  the  total 
cost  of  lecture  revision.  Because  it  is  difficult  to  determine  how  many 
lectures  needed  revision,  the  percent  of  workbooks  revised  was  assumed 
to  reflect  the  number  of  lectures  needing  revision.  A  percentage  of  the 
wages  of  VP-30  graphics  people  and  their  operating  budget  were  included 
in  the  lecture  revision  costs  for  VP-30  because  the  graphics  division 
exists  in  part  to  support  their  lectures. 

It  could  be  argued  that  using  the  hourly  wage  of  the  instructors 
making  the  revisions  is  not  vadid  because  the  military  instructors  are 
being  paid  regardless  of  how  many  changes  they  make.  While  that 
observation  is  true,  a  cost  measurement  must  be  made  in  order  to  compare 
the  opportunity  costs  of  one  medium  with  another.  If  the  instructor  was 
not  doing  revisions,  he  could  have  been  engaged  in  some  other  productive 
or  useful  activity,  and  this  is  in  fact  the  cost  associated  with  the  changes. 

2.  Media  Portability 

Another  attribute  of  a  training  medium  that  is  often  desirable  is 
one  of  portability  if  training  must  be  provided  away  from  the  usual 
training  site.  Within  aviation  commands,  the  aircraft  are  occasionally 
upgraded  to  a  newer  model.  When  a  squadron  transitions  to  a  different 
model  of  aircraft,  the  fleet  squadron  must  train  its  aircrewmen  on  the  new 
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equipment  and  software.  The  training  required  for  the  transition  can  be 
done  at  either  the  Fleet  Replacement  Squadron  by  bringing  all  aircrewmen 
to  the  training  site,  or  at  the  operational  squadron's  location. 

In  the  case  of  workbooks  and  lectures,  both  media  are  portable 
but  transferring  these  incurs  different  costs.  The  workbooks  can  be 
bundled  up  and  mailed  off,  or  the  instructors  can  catch  the  next  plane. 
The  cost  of  bringing  training  to  the  fleet  squadron  idses  quickly  if  many 
instructors  are  required  to  conduct  the  training.  A  problem  with  sending 
many  instructors  away  from  their  training  command  is  that  the  instructors 
remaining  behind  must  fill  in  for  the  absent  instructors.  Filling  in  for  the 
absent  instructors  may  result  in  a  tired  staff  and  dissatisfaction  resulting 
from  overwork.  A  training  program  structured  around  workbooks  may  cost 
less  to  transport  because  workbooks  do  not  require  instructors. 

A  cost  measure  of  media  portability  has  been  developed  by  using 
a  scenario  in  which  a  four  week  training  program  is  assumed  to  take  place 
at  the  operational  squadron's  location.  The  assumptions  are  that  under 
the  workbook  program  two  instructors  per  aircrew  position  are  required 
along  with  the  workbooks,  and  that  three  instructors  are  required  to  do 
the  training  using  the  lecture  method.  The  costs  are  computed  by 
multiplying  the  instructor's  hourly  wage  times  the  number  of  hours  of 
instruction,  adding  travel  expenses  and  summing  these  amounts  over  all 
aircrew  positions.  The  costs  could  also  be  determined  for  a  scenario 
where  the  fleet  aircrewmen  report  to  the  Fleet  Replacement  Squadron  for 
the  training. 
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3.  Staff  Absence,  Student  Loading,  Standardization,  and  Turnover 

Other  cost  advantages  occur  because  some  instructional  programs 
can  function  better  than  others  in  the  face  of  instructor  absences.  Navy 
training  programs  operate  under  rigid  time  lines  with  very  little  time 
allowed  for  making  up  missed  training  events.  Instructor  absence  cannot, 
therefore,  be  easily  accommodated.  A  medium  such  as  workbooks,  which 
presents  the  training  material  without  the  presence  of  an  instructor,  has 
a  cost  advantage  over  a  lecture  method  that  would  require  a  replacement 
instructor  to  keep  the  training  program  on  schedule. 

However,  the  production  workers  are  critical  to  the  revision 
process  of  the  workbook  medium.  If  staffing  level  is  at  the  level  required 
for  the  useful  flow  of  changes,  someone's  absence  for  a  lengthy  period  or 
a  job  unfilled  because  of  a  hiring  freeze,  can  lead  to  a  large  revision 
backlog  and  long  periods  with  inaccurate  workbooks.  The  lecture  medium 
requires  few  production  workers,  and  those  do  not  hold  critical  positions, 
so  their  absence  has  little  affect  on  a  lecture  program. 

Some  training  programs  may  also  be  faced  with  surges  or  declines 
in  student  loading.  The  workbook  medium  can  easily  deal  with  a 
temporary  surge  in  student  loading  as  long  as  the  books  are  available. 
Printing  extra  workbooks  is  relatively  inexpensive  and  should  not  require 
too  much  lead  time.  With  lectures,  when  increased  student  loading  exceeds 
the  class  capacity  additional  instructors  are  required.  Because  instructors 
have  numerous  other  responsibULties  in  the  training  command,  the  added 
demand  imposes  a  cost  because  of  the  reduced  productivity  in  other  areas 
of  responsibility.  Student  loadmg  in  P-3  training  is  typically  stable  so  an 
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operating  cost  resulting  from  changes  in  student  loading  was  not 
determined,  however,  in  a  qualitative  sense  workbooks  should  have  a  cost 
advantage  if  students  increase  or  decrease. 

The  ease  with  which  a  training  program  can  be  standardized  will 
vary  from  one  program  to  another  depending  on  the  media  used  in  the 
training  program  among  other  things.  Poor  standardization  in  the  training 
program  may  lead  to  a  failure  to  meet  the  objectives  for  which  the 
training  program  was  established.  The  gap  between  what  a  squadron  is 
really  teaching  and  what  should  be  taught  as  determined  by  the  training 
objectives  has  an  opportunity  cost  associated  with  it.  That  cost  is 
manifested  in  the  extra  effort  required  by  the  fleet  squadron  to  bring  the 
graduate  to  the  level  he  should  have  been  upon  reporting. 

People  are  probably  the  source  of  the  greatest  variability  in 
training  courses.  Student  learning  can  vary  greatly  depending  on 
instructor  knowledge,  instructor  teaching  ability,  and  student  classroom 
attention.  Instructor  knowledge  and  teaching  ability  can  vary  greatly 
from  one  instructor  to  another.  Some  instructors  are  far  better  at 
recognising  when  a  student  is  having  difficulty,  and  at  figuring  out  other 
ways  to  present  the  material.  Students  have  their  good  and  bad  days, 
and  their  attention  in  the  class  is  not  always  the  same.  Any  material 
presented  by  the  instructor  during  a  student's  attention  lapse  is  lost  to 
the  student. 

A  workbook  presents  information  the  same  way  every  time.  This 
reduces  variability  due  to  the  instructor.  With  a  workbook,  a  student  may 
continuously  gauge  his  performance  relative  to  the  exact  requirements  of 
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the  course  using  the  quizzes  provided.  Poor  performance  in  certain  areas 
prompts  a  student  to  go  back  and  study  the  areas  that  were  not  learned 
well  enough.  Students  completing  the  workbooks  ahead  of  schedule  may 
have  the  opportunity  to  expand  their  knowledge  of  advanced  material.  An 
attention  lapse  does  not  have  the  same  affect  as  in  the  lectures  because 
the  student  still  has  the  course  material  in  front  of  him.  By  design,  the 
workbook  method  with  its  fixed  presentation  and  more  structured  change 
mechanism,  is  better  standardized  than  lectures. 

Staff  turnover  is  a  problem  incurred  regardless  of  the  media 
used  for  instruction,  but  there  may  be  differences  in  the  media  that  makes 
one  less  costly  in  terms  of  staff  turnover  than  the  other.  With  rapid  staff 
turnover,  an  instructor  is  often  transferred  shortly  after  developing  his 
expertise.  A  longer  time  in  a  position  is  usually  accompanied  by  a  greater 
proficiency  within  the  position.  Since  aircrew  training  is  accomplished 
with  military  instructors,  rapid  turnover  is  common  as  people  routinely 
leave  the  training  organization  after  several  years  of  duty. 

Instructors  involved  with  a  lecture  method  of  instruction  spend 
several  weeks  or  more  learning  to  present  lectures,  an  activity  not 
required  for  a  workbook  method  of  instruction.  However,  the  greatest 
cost  arising  from  rapid  turnover  in  an  ISD  developed  training  program 
may  be  from  the  changes  that  are  made  to  the  training  program.  People 
with  experience  and  an  in-depth  understanding  of  the  ISD  process  from 
which  the  training  program  was  developed  are  rare.  Someone  without 
sufficient  knowledge  may  make  changes  to  the  training  program,  the 
workbooks,  or  the  lectures,  that  are  not  consistent  with  the  objectivts  of 
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the  training  program.  The  cost  of  rapid  turnover  may  then  depend  on 
how  much  damage  was  done  by  those  who  made  the  incorrect  or 
unnecessary  changes.  In  the  P-3  training  program,  the  workbooks  require 
less  instructor  preparation  time,  but  damaging  changes  to  the  training 
program  could  be  made  by  the  staff  in  either  method  of  instruction. 

There  may  be  other  attributes  that  are  important  to  evaluate 
when  considering  different  training  media.  When  evaluating  media 
alternatives,  conventional  instruction  is  usually  thought  of  as  the  baseline 
in  the  comparison.  Instructor  or  student  prejudice  may  make  the  use  of 
an  alternative  difficult.  A  teaching  method  may  not  be  readily  accepted 
by  either  the  instructors  or  the  students  because  it  is  different  or 
unfamiliar.  The  lecture  method  of  teaching  is  something  that  edl  students 
grew  up  with  and  usually  expect  when  entering  the  training  command. 
The  instructors  may  have  a  natural  bias  towards  lectures  because  they 
envision  lectures  as  the  proper  way  to  teach  and  are  unfamiliar  with  other 
methods. 

In  aviation  training,  most  instructional  hours  are  devoted  to 
teaching  technical  material.  It  is  a  very  tedious  process  to  learn  the 
matericd  with  a  static  medium  such  as  workbooks.  Learning  through 
lectures  seems  to  ease  this  burden.  As  mentioned  earlier,  this  was  the 
reason  VP-31  instituted  a  learning  center  instructor  in  1982.  This 
indicates  that  the  costs  of  a  workbook  program  must  also  include  the  costs 
of  instructor  interaction  required  to  maintain  similar  levels  of  training. 
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V.  DATA  AND  ANALYSIS 


This  chapter  compares  the  costs  of  the  workbook  and  lecture  training 
programs  in  use  at  VP-31  and  VP-30.  Tables  5.13,  5.14,  and  5.15  present 
the  costs  for  the  comparison  in  the  same  format  as  the  cost  model 
described  in  chapter  four,  and  the  other  tables  in  this  chapter  show  how 
those  costs  were  derived. 

Table  5.1  shows  the  approximate  annual  student  enrollment  for  the 
different  aircrew  positions.  The  student  loading  was  determined  by 
calculating  the  fleet  squadron’s  manpower  demands  for  the  twelve 
operational  squadrons  supported  by  its  Fleet  Replacement  Squadron.  The 
calculations  were  based  on  each  sq’’adron  requiring  three  pilots,  two  naval 
flight  officers,  two  flight  engineers,  two  acoustic  operators,  one  non¬ 
acoustic  operator,  one  ordnanceman,  and  one  inflight  technician  per  P-3 
crew.  Each  operational  squadron  was  assumed  to  require  ten  crews.  The 
length  of  tour  used  in  the  calculation  was  three  years  for  flight  engineer, 
ordnanceman,  and  inflight  technician,  and  three  and  a  half  years  for  aU 
others. 

The  military  salaries  used  in  these  computations  are  shown  in  Table 
5.2  and  were  obtained  from  NAVCOMPTNOTE  7041  of  8  NOV  1989  which  lists 
the  composite  standard  rates  for  military  people.  These  rates  are 
prescribed  for  use  when  the  costs  of  military  services  are  determined  on 
a  statistical  basis.  Civilian  salaries  were  obtained  from  the  general 
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TABLE  5.1.  FLEET  REPLACEMENT  SQUADRON  STUDENT  LOADING 


Aircrew  Position 

Annual  Number  of  Students 

Pilot 

120 

Naval  Flight  Officer 

80 

Flight  Engineer 

80 

Acoustic  Operator 

69 

Non-Acoustic  Operator 

35 

Ordnanceman 

40 

Inflight  Technician 

40 

schedule  salary  table  number  75,  authorized  by  Executive  Order  12698  of 
December  23,  1989,  and  were  multiplied  by  a  fringe  benefit  factor  of  1.184. 
The  addition  of  the  fringe  benefit  factor  provides  a  better  estimate  of  the 
actual  costs  of  employing  civilian  workers.  For  military  and  civilian 
workers,  the  hourly  wage  was  based  on  2080  working  hours  per  year  as 
specified  in  the  NAVCOMPT  manual. 


TABLE  5.2.  MILITARY  AND  CIVILIAN  WAGE  RATES 


Rank/Rate/Grade 

Annual  Cost 

Hourly  Cost 

03 

64,336 

$30.93 

02 

49,450 

23.77 

01 

37,394 

17.98 

E-7 

44,152 

21.23 

E-6 

37,085 

17.83 

E-5 

30,760 

14.79 

E-4 

25,530 

12.27 

E-3 

21,621 

10.39 

GS-12 

43,830 

21.07 

GS-11 

36,570 

17.58 

GS-9 

30,226 

14.53 

GS-7 

24,708 

11.88 

GS-5 

19,949 

9.59 

GS-4 

17,830 

8.57 

GS-3 

15,883 

7.64 
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Table  5.3  provides  estimates  of  the  hours  per  course  spent  by 
students  and  instructors  in  the  two  instri;''tional  programs.  The  estimates 
were  arrived  at  using  the  curriculum  descriptions  for  each  of  the  seven 
aircrew  training  tracks  at  the  Fleet  Replacement  Squadrons.  For  VP-31, 
the  student  learning  center  hours  represent  the  scheduled  hours  that  the 
students  spent  studying  the  workbooks  and  taking  the  unit  tests.  At 
VP -31,  an  instructor  is  required  to  be  present  i”.  the  learning  center 
during  the  time  that  students  are  assigned  there.  The  reason  that  the 
instructor  learning  center  hours  are  less  than  the  student  hours  is 
because  of  an  overlap  in  classes.  For  example,  four  different  classes  of 
Naval  Flight  Officers  are  at  different  stages  of  VP-Sl's  training  program 
simultaneously.  If  students  from  each  of  the  four  classes  are  scheduled 
for  the  learning  center  on  the  same  day,  only  one  instructor  is  required 
for  all  four  classes.  Learning  center  usage  at  VP-30  is  mostly  for  self- 
study  of  reference  materials  other  than  workbooks,  and  instructors  are  not 
usually  assigned,  However,  instructors  are  still  required  to  administer 
tests.  The  student's  rank  or  rate  which  is  needed  for  later  cost 
comparisc  .s,  is  indicated  in  parenthesis. 

The  hours  shown  for  flight  engineer,  ordnanceman,  and  inflight 
technician  do  not  represent  the  total  hours  of  training  which  these  aircrew 
positions  receive  prior  to  graduation.  The  training  program  for  these 
positions  is  divided  among  different  commands  making  it  difficult  to 
compare  training  hours  with  the  given  media.  The  hours  shown  under 
VP-31  are  the  hours  spent  in  lectures  or  the  learning  center  at  VP-31. 
The  training  program  at  VP-30,  for  the  same  aircrew  positions,  was  so 
different  that  a  direct  comparison  could  not  be  made.  VP-31's  training 
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TABLE  5.3.  LEARNING  CENTER  AND  LECTURE  HOURS  PER  COURSE 

AT  VP-31  AND  VP-30 


Aircrew  Position 

VP- 

31 

Lecture 

VP-30 

Learning  Cntr. 

Lecture 

Pilot  (01) 

400 

Inst. 

134 

75 

Stvidsnt  JnsL. 

249  18 

126 

Naval  Flight  Off. (01) 

258 

86 

114 

228 

27 

186 

Plight  Eng.  (E-5) 

88 

16 

224 

88 

16 

224 

Acoustic  (E-4) 

239 

80 

219 

200 

65 

166 

Non -Acoustic  (E-4) 

275 

92 

282 

175 

60 

288 

Ordnanceman  (E-3) 

2 

0 

88 

2 

0 

88 

Inflight  Tech.  (E-4) 

28 

20 

19 

28 

20 

19 

hours  were  used  on  both  sides  of  the  equation  which  provided  the  ability 
to  count  the  hours  associated  with  the  workbooks  at  VP-31,  and  later,  the 
costs.  The  training  conducted  at  VP-30  was  assumed  to  be  conducted  with 
lectures.  Because  the  flight  engineer  and  ordnanceman  training  tracks  at 
VP-31  use  more  lectures  than  workbooks,  this  assumption  could  bias  the 
study.  If  the  lecture  method  is  mo^e  expensive,  the  assumption  increases 
the  costs  of  VP-31's  training,  if  the  lecture  method  is  less  expensive,  it 
will  cause  VP-31's  costs  to  be  lower. 

».nnual  hours  for  each  trainu.g  program  were  computed  by  multiplying 
the  hours  per  course  by  the  number  of  courses  annually.  VP-31  operates 
its  training  program  using  a  24  day  schedule.  Each  24  days  is  called  a 
phase.  Most  of  the  students  in  the  different  aircrew  tracks  require  four 
phases,  or  96  training  days,  to  complete  their  training.  At  VP-31,  classes 
begin  after  the  rotation  of  a  phase,  or  every  24  days.  VP-30's  training 
program  is  90  training  days  long  and  is  divided  into  three  phases,  each 
phase  being  30  days  long.  Dividing  the  trainir--;  program  into  three  or 
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four  phases  means  that  VP-31,  with  the  four  phase  system,  teaches  ten 
new  classes  per  year.  VP-30  with  the  three  phase  system  teaches  eight 
new  classes  per  year.  The  number  of  classes  per  year  is  needed  for  later 
calculations. 

Since  the  number  of  students  is  identical  in  both  training  programs, 
operating  with  fewer  classes  each  year,  as  done  at  VP-30,  increases  the 
number  of  students  in  each  class.  An  interesting  result  of  having  eight 
instead  of  ten  classes  annually  is  the  reduced  instructor  hours.  For 
example,  ten  pilot  classes  at  VP-31  will  ’"‘quire  2,09t  instructor  hours 
annually  (learning  center  instructor  plus  lecture  instructor  times  ten).  If 
VP-31  changed  its  schedule  to  a  ,hree  phase  system  with  each  phase  being 
32  days  long,  just  under  eight  classes  could  be  taught  annually  requiring 
roughly  1,672  pilot  instructor  hours,  or  418  less  hours.  Changing  to  a 
three  phase  system  would  save  818  officer  and  2,080  enhsted  instructor 
hours  annually  over  all  training  tracks.  Changing  from  ten  to  eight 
classes  annuedly  wV  change  the  pilot  class  size  from  12  to  15  students  and 
the  naval  flight  officer  class  size  from  eight  to  ten.  Changes  of  this 
magnitude  are  negligible,  and  should  not  affect  the  quality  of  training  or 
the  instructor  effort  required  to  teach  the  class.  Larger  differences, 
caused  by  eliminating  one  of  the  Fleet  Replacement  Squadrons,  would 
increase  class  size  to  around  thirty.  Teaching  thirty  students  would 
create  a  sig’uficantly  heavier  demand  on  instructor  time,  and  create  new 
problems  not  seen  with  the  change  from  12  to  15.  One  cost  of  changing 
to  a  three  phase  system  is  the  loss  in  productivity  of  the  students 
because  the  students  will  be  delayed  in  reporting  to  their  operational 


46 


squadrons  by  the  same  amount  of  delay  in  starting  the  training  program. 
Another  problem  is  that  the  program  set  up  with  eight  courses  per  year 
must  schedule  more  aircraft  trainers  and  aircraft  flights  during  the  flight 
phase  of  instruction  to  accommodate  the  increased  number  of  students. 
Thus,  while  a  change  from  12  to  15  students  may  be  negligible,  the  same 
cannot  be  said  of  an  increase  from  12  to  30. 

In  order  to  determine  the  cost  of  media  development  and  the  cost  of 
maintaining  media  currency,  an  estimate  of  the  time  it  takes  to  develop  or 
revise  a  workbook  or  lecture  is  needed.  A  survey  completed  by  several 
VP-31  track  managers  and  discussion  with  the  VP-31  educational  specialist 
provided  the  information  used  to  make  these  estimates.  Table  5.4  shows 
the  time  needed  for  major  or  minor  revisions  to  workbooks  and  the  time 
taken  annually  for  workbook  review  averaged  across  all  seven  training 
tracks.  Major  revisions  required  because  of  substantial  equipment  changes 
usually  require  a  complete  remake  of  the  workbook.  Minor  revisions  may 
require  up  to  several  pages  of  changes.  An  annual  review  is  the  time 
spent  reviewing  a  workbook  without  having  to  make  changes.  Roughly  33 
percent  of  the  850  workbooks  in  the  system  require  major  revisions 
annually  and  66  percent  require  minor  revisions.  Twenty  two  percent  of 
the  workbooks  are  reviewed  annually  with  no  changes.  These  percentages 
exceed  100  percent  because  some  workbooks  require  more  than  one 
revision  annually.  The  total  hours  shown  represent  the  hours  required 
for  all  training  tracks  and  come  from  a  mix  of  different  wage  earners  in 
the  revision  process.  A  breakdown  of  the  hours  required  for  the  different 
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functions  in  the  revision  process  is  shown  to  provide  an  idea  of  the  effort 
involved  in  workbook  revisions. 

The  annual  cost  for  revisions  is  shown  in  Table  5.5.  Computing  this 
cost  required  determining  the  wages  of  all  people  in  the  revision  process, 
determining  the  hours  required  to  make  the  revision,  and  determining  how 
many  revisions  are  made  within  the  different  training  tracks. 

The  revision  costs  associated  with  subject  matter  experts  and  track 
managers  were  calculated  as  follows.  A  total  of  850  workbooks  are 
required  in  the  training  program  to  teach  the  information  associated  with 
all  the  different  models  of  P-3's.  Of  these,  350  are  used  in  the  pilot  and 
naval  flight  officer  tracks.  For  major  revisions  in  the  officer  tracks,  the 
350  workbooks  were  multiplied  by  33.67  percent  (the  number  of  major 
revisions  annually).  This  number  was  multiplied  by  the  average  hours 
required  for  major  revisions  (27  hours)  and  by  the  hourly  wage  rate  for 
officers  at  the  C3  level  ($30.95).  The  com.putation  for  the  enlisted  tracks 
was  done  in  a  similar  manner.  The  remaining  500  enlisted  workbooks  (850 
total  workbooks  -  350  officer  workbooks)  was  multiplied  by  33.67  percent, 
by  the  revision  time  of  27  hours,  and  by  the  E-6  wage  race  of  $17.83. 
The  officer  and  enlisted  revision  costs  were  then  added  together. 

Production  workers  in  the  ISD,  graphics,  and  printing  departments, 
are  employed  to  keep  the  workbooks  current.  Because  these  departments 
engage  in  some  activities  not  related  to  workbook  revision,  only  a 
percentage  of  their  annual  sala  ;ies,  determined  during  supervisor 
interviews,  were  used.  One  hundred  percent  of  the  ISD  sedaries,  60%  of 
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VP-31's  graphics  salaries,  and  90%  of  the  salaries  of  the  graphics  and 
printing  departments  under  the  Fleet  Aviation  Specialized  Operational 
Training  Group  (FASO)  were  used.  These  salary  percentages  were  divided 
between  the  major  and  minor  revisions  based  on  the  estimated  effort 
reqxiired  for  each  type  of  revision. 


TABLE  5.4.  TIME  REQUIRED  FOR  MAJOR  AND  MINOR  REVISIONS, 
AND  ANNUAL  REVIEW  OF  WORKBOOKS 


Position 

Average 

Hours/Workbook 

Total  Hours 

Per /Year 

MAJOR  REVISION 

Subject  Matter  Expert 

27.0 

8182 

Track  Manager 

18.8 

5707 

Educational  Specialist 

4.0 

1212 

Editor 

4.0 

1212 

Typists 

16.0 

4848 

Graphics 

8.5 

2576 

Print 

2.0 

606 

MINOR  REVISION 

Subject  Matter  Expert 

3.8 

2202 

Track  Manager 

4.8 

2777 

Educational  Specialist 

2.0 

1149 

Editor 

1.5 

862 

Typists 

7.0 

4021 

Graphics 

4.2 

2394 

Print 

2.0 

1149 

ANNUAL  REVIEW 

Subject  Matter  Expert 

1.7 

340 

Track  Manager 

2.0 

408 
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TABLE  5.5.  COSTS  FOR  MAJOR  AND  MINOR  REVISIONS, 
AND  ANNUAL  REVIEW  OF  WORKBOOKS 


Position 

Cost  For 

Revisions 

MAJOR  REVISION 

Subject  Matter  Expert 

$179,460 

Track  Manager 

124,960 

ISD  Department 

84,750 

Graphics 

71,340 

Print 

48,000 

MINOR  REVISION 

Sub  Total 

$508,510 

Subject  Matter  Expert 

$47,890 

Track  Manager 

60,490 

ISD  Department 

72,190 

Graphics 

60,770 

Print 

70,080 

ANNUAL  REVIEW 

Sub  Total 

$311,420 

Subject  Matter  Expert 

$7610 

Track  Manager 

8950 

Sub  Total 

$16,560 

TOTAL 

$837,310 

Original  workbook  development  was  a  major  undertaking  during  the 
instructional  system  development  (ISD)  process.  To  estimate  the  workbook 
development  costs  shown  in  Table  5.6,  the  development  hours  per  workbook 
for  the  subject  matter  experts  and  track  managers  were  assumed  to  be  the 
same  as  the  hours  required  for  a  major  workbook  revision.  The 
development  cost  estimate  is  based  on  developing  850  workbooks,  each  of 
which  has  35  pages.  The  computations  are  identical  to  the  computations 
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for  major  workbook  revision,  except  that  100  percent  of  the  workbooks 
underwent  development.  These  hours,  and  the  costs  determined  from  the 
hours,  probably  underestimate  actual  development  time  and  cost  because 
a  development  process  usually  requires  many  more  iterations  to  produce 
the  print  ready-copy  than  a  revision  process  because  the  workbook  form 
has  already  been  decided.  When  making  revisions,  less  effort  is  required 
to  organize  the  material  for  orderly  presentation. 

The  development  cost  estimate  for  ISD,  graphics,  and  printing,  cannot 
be  computed  by  using  a  percentage  of  annual  costs  as  done  for  revision 
costs,  because  more  than  one  year  would  be  required  for  development. 
Here,  two  years  of  total  annual  salaries  for  ISD,  graphics,  and  printing  are 
assumed.  A  two  year  cost  estimate  still  probably  underestimates  these 
development  costs.  The  actual  original  curriculum  development  costs  were 
$7,962,000  as  presented  by  the  Thode-Walker  study  [Ref.  17].  But,  these 
costs  represent  the  development  costs  for  the  entire  curriculum  and 
include  developing  other  course  media  such  as  videotapes  and  aircraft 
events. 


TABLE  5.6.  COST  OF  WORKBOOK  DEVELOPMENT 


Position 

Total  Development 
Time  (Hrs) 

Total  Development 
Cost 

Subject  Matter  Expert 

22,950 

$532,900 

Track  Manager 

15,980 

371,060 

ISD  Department 

20,400 

313,870 

Graphics 

7,225 

370,240 

Print 

1,700 

262,410 

TOTAL  $1,850,480 
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Lecture  development  and  revision  time  was  more  difficult  to  measure 
because  these  times  vary  considerably  depending  on  the  technical  nature 
of  the  lecture  and  the  expertise  of  the  person  developing  it.  From 
discussions  with  instructors  and  responses  to  the  author's  survey,  12 
hours  was  chosen  as  the  estimate  for  the  time  required  to  develop  one 
hour  of  lecture.  This  same  time  was  used  as  an  estimate  of  the  time 
required  for  a  major  revision  for  one  hour  of  lecture.  Two  hours  of 
revision  time  was  used  as  the  time  required  for  a  minor  revision.  The 
revision  frequency  was  assumed  to  be  that  used  for  the  workbooks,  i.e., 
33%  of  the  lectures  had  major  revisions  annually  and  68%  had  minor 
revisions  annually.  The  time  and  cost  of  lecture  development  and  revision 
are  shown  in  Tables  5.7  and  5.8. 


TABLE  5.7.  COST  OF  LECTURE  DEVELOPHENT 


VP- 

-31 

VP- 

-30 

Development  Development 

Development  Devdopment 

Position 

Time  (Hrs) 

Cost 

Time  (Hrs) 

Cost 

Pilot 

900 

$27,840 

1520 

$47,020 

Naval  Flight  Off. 

1368 

42,320 

2240 

69,290 

Flight  Eng. 

2688 

47,930 

2688 

47,930 

Acoustic 

2628 

46,850 

1992 

35,520 

Non-Acoustic 

3384 

60,340 

3456 

61,620 

Ordnanceman 

1056 

18,830 

1056 

18,830 

Inflight  Tech. 

228 

4,070 

228 

4,070 

TOTAL 

$248,180 

$284,280 
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TABLE  5.8.  COSTS  OF  LECTURE  REVISIONS 
BY  TRAINING  TRACK 


Instructor 

Major  Revision 
Hours  Dollars 

Minor 

Hours 

Revision 

Dollars 

VP-31 

Pilot  (03) 

297 

$9,190 

102 

$3,160 

Naval  Flight  Off.  (03) 

451 

13,950 

155 

4,800 

FUght  Eng.  (E-6) 

887 

15,820 

304 

5,420 

Acoustic  (E-6) 

867 

15,460 

297 

5,300 

Non-Acoustic  (E-6) 

1116 

19,900 

383 

6,830 

Ordnanceman  (E-6) 

348 

6,210 

119 

2,130 

Inflight  Tech.  (E-6) 

75 

1,340 

25 

450 

TOTAL 

$81,870 

$28,090 

VP -30 

Pilot  (03) 

499 

$15,440 

171 

$5,290 

Naval  Flight  Off.  (03) 

736 

22,770 

253 

7,830 

Fhght  Eng.  (E-6) 

887 

15,820 

304 

5,420 

Acoustic  (E-6) 

657 

11,720 

225 

4,020 

Non-Acoustic  (E-6) 

1140 

20,330 

391 

6,980 

Ordnanceman  (E-6) 

348 

6,210 

119 

2,130 

InfUght  Tech.  (E-6) 

75 

1,340 

25 

450 

TOTAL 

$93,630 

$32,120 

The  graphics  and  printing  departments  exist  to  support  revisions  to 
the  workbook  media  so  their  operating  budgets  were  •  'luded  in  the  cost 
estimate  for  media  currency  and  are  shown  in  Table  5.9.  Because  some  of 
their  functions  are  not  workbook  related,  only  ninety  percent  of  PASO's 
graphics  and  printing  department  budgets  were  used.  VP-31's  graphics 
department  budget  was  not  included  because  it  would  exist  even  if  the 
workbooks  were  not  in  the  training  program,  just  as  VP-30's  graphics 
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department  exists.  The  percentages  were  determined  through  interviews 
with  the  respective  supervisors. 


TABLE  5.9.  ANNUAL  BUDGET  OF  DEPARTMENTS 
ASSOCIATED  WITH  WORKBOOKS 


Department 


Annual  Dollars 


Graphics  $11800 

Print  9720 


TOTAL  $21,520 


The  labor  cost  of  each  course,  which  was  estimated  as  the  time  that 
the  instructors  and  students  are  involved  with  the  media,  is  shown  in 
Table  5.10  and  5.11.  The  costs  shown  in  the  these  tables  were  determined 
by  multiplying  the  wage  rate  for  the  instructors  and  students  by  the 
hours  spent  with  the  media  as  shown  earlier  in  Table  5.3. 


TABLE  5.10.  COSTS  PER  COURSE  FOR  INSTRUCTOR  TIME  WITH  MEDIA 


VP-31  VP-30 


Instructor 

LC* 

LECT* 

LC 

LECT 

Pilot 

$4150 

$2320 

$560 

$3900 

Naval  Flight  Off. 

2660 

3530 

840 

5760 

Flight  Eng. 

290 

4000 

290 

4000 

Acoustic 

1430 

3910 

1160 

2960 

Non -Acoustic 

1640 

5030 

1070 

5140 

Ordnanceman 

0 

1570 

0 

1570 

Inflight  Tech. 

360 

340 

360 

340 

Sub  Total 

$10,530 

$20,700 

$4,280 

$23,670 

TOTALS  (LC+LECT) 

$31,230 

$27,950 

*  LC  stands  for  learning  center,  LECT  stands  for  lecture. 

TABLE  5.11.  COSTS  PER  COURSE  FOR  STUDENT’S  TIME  WITH  MEDIA 


Instructor 

VP- 

LC 

•31 

LECT 

VP- 

LC 

30 

LECT 

Pilot 

$86,310 

$16,190 

$67,160 

$33,990 

Naval  Flight  Off. 

37,120 

16,400 

41,000 

33,450 

Flight  Eng. 

10,420 

26,510 

13,020 

33,130 

Acoustic 

20,240 

18,550 

21,170 

17,570 

Non-Acoustic 

11,810 

12,120 

9,400 

15,460 

Ordnanceman 

90 

3,660 

110 

4,580 

Inflight  Tech. 

1,380 

940 

1,720 

1,170 

Sub  Total 

$167,370 

$94,370 

$153,580 

$139,350 

TOTALS  (LC+LECT) 

$261,740 

$292,930 

The  cost  of  portability,  shown  in  Table  5,12,  was  determined  using 
a  scenario  that  assumed  instruction  was  required  at  an  out-of-town 
training  site.  For  this  scenario,  the  workbook  method  of  instruction  was 
assumed  to  require  two  instructors  per  aircrew  position  and  the  lecture 
method  to  require  three.  Under  these  assumptions,  four  officer  and  ten 
enlisted  instructors  were  required  to  teach  with  the  workbook  method, 
while  six  officer  and  15  enlisted  were  required  for  the  lecture  method. 
Training  was  assumed  to  take  one  month,  with  five  working  days  per  week, 
or  a  total  of  160  training  hours  per  instructor.  The  training  time  was 
multiplied  by  the  instructors'  wages.  A  daily  per  diem  rate  of  $24  per 
day  per  instructor  was  added. 
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TABLE  5.12.  COST  OF  PORTABILITY 


VP-31 

No.  of  Instructors 

Dollars 

VP-30 

No.  of  Instructors 

Dollars 

4  Officers  (03) 

$22,680 

6  Officers  (03) 

$34,020 

10  Enlisted  (E-6) 

35,730 

15  Enlisted  (E-6) 

53,600 

TOTAL 

$58,410 

$87,620 

The  plant  and  equipment  costs  for  VP-31  are  shown  in  Table  5.13. 
These  costs  are  the  purchase  costs  of  the  eaviipment  needed  to  support 
changes  to  the  workbooks  and  are  measured  in  current  dollars.  Equipment 
costs  to  support  changes  to  lectures  wer?  assumed  to  be  negligible. 

Table  5.14  shows  the  combined  opportunity  costs  estimate  associated 
with  developing  the  workbooks  and  lectures  at  VP-31  and  the  lectures  at 
VP-30.  The  development  costs  for  lectures  at  VP-31  and  VP-30  are  nearly 
the  same  which  indicates  that  a  large  number  of  lectures  have  been 
instituted  at  VP-31.  The  high  workbook  de^■elopment  cost  is  the  result  of 
the  great  amount  of  time  required  to  develop  textbooks. 

Development  costs  are  sunk  costs  and  of  importance  only  if  a  new 
development  effort  is  contemplated.  Table  5.15  compares  the  present 
operating  costs  based  on  the  opportunity  costs  associated  with  the  actions 
req\iired  to  support  either  medium.  The  production  process  costs  shown 
are  fairly  similar,  indicating  that  both  training  programs  spend  about  the 
same  amount  of  time  presenting  the  theoretical  information.  The  large 
difference  in  the  currency  costs  is  due  to  the  effort  required  of  the 
subject  matter  experts  and  track  managers,  and  the  extra  manpower 
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required  in  the  ISD,  graphics,  and  printing  departments,  to  revise  the 
workbooks.  The  cost  differences  in  portability,  as  shown  in  Table  5.15, 
hidicate  that  the  workbooks  have  a  portability  advantage.  This  portability 
advantage  results  from  the  fact  that  workbooks  can  substitute  for 
instructors. 


TABLE  5.13.  PLANT  AND  EQUIPMENT  COSTS  AT  VP-31 


Department 

Equipment  Cost 

ISD 

$37,575 

Graphics 

45,700 

Printing 

57,600 

TOTAL 

$140,875 

TABLE  5.14.  DEVELOPMENT  COSTS 


Media 

VP-31 

VP-30 

Lectures 

$248,180 

$284,280 

Workbooks 

1,850,480 

0 

TOTAL 

$2,098,660 

$284,280 

TABLE 

5.15.  ANNUAL 

OPERATING  COSTS 

Measure 

VP-31 

VP-30 

Production 

Process 

$2,929,700 

$2,567,040 

Currency 

968,790 

125,750 

Portability 

29,210 

43,810 

TOTAL 

$3,927,700 

$2,736,600 
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The  costs  from  Tables  5.13,  5.14,  and  5.15,  were  used  to  determine  the 
life-cycle  costs  of  the  training  programs  over  a  ten  year  period.  These 
costs  are  shown  in  Table  5.16.  To  compute  the  life-cycle  costs,  50%  of 
the  plant  and  equipment  costs  were  assumed  to  depreciate  to  zero  and 
need  replacing  at  the  end  of  ten  years.  The  replacement  costs  of  plant 
and  equipment  were  assumed  to  increase  at  an  annual  inflation  rate  of  4%. 
Development  costs  were  assumed  to  be  a  one  time  cost  with  no  replacement 
costs.  The  annual  operating  costs  are  composed  mostly  of  salaries,  and 
these  salaries  were  assumed  to  increase  at  an  annual  rate  of  3%.  For  the 
net  present  value  compv  Utions,  a  discount  rate  of  9%  was  used.  To 
account  for  inflation,  the  discount  rate  was  combined  with  the  inflation 
rate  using  the  formula 

nominal  rate  (i)  =  real  rate  (r)  +  inflation  rate  (tc)  +  m. 


TABLE  5.16.  TEN  YEAR  LIFE-CYCLE  COSTS 


Cost  Category 

Ten 

VP-31 

Year  Costs 

VP-30 

Plant  and  Equipment 

ISD 

$45,510 

$0 

Graphics 

55,360 

0 

Printing 

69,770 

0 

Development  Costs 

Lectures 

248,180 

284,280 

Workbooks 

1,850,480 

0 

Production  Process 

17,433,770 

15,275,690 

Currency 

5,764,980 

748,300 

Portability 

173,820 

260,700 

TOTAL 

$25,641,870 

$16,568,970 

58 


Scheduled  aircraft  transitions  are  expected  to  reduce  the  number  of 
P-3  aircraft  models  used  at  operational  squadrons  from  seven  to  three  by 
1992.  The  reduced  number  of  models  could  reduce  the  number  of 
workbooks  from  roughly  850  to  550.  A  substantial  savings  in  opportunity 
cost  will  be  realized  with  fewer  workbooks  because  of  reduced  revision 
costs.  Further  savings  in  workbook  revision  will  result  from  a  workbook 
design  change  presently  in  progress.  The  design  change  modifies  the 
workbooks  so  that  frequently  changing  material  will  be  removed  and  the 
students  will  be  referred  to  reference  manuals  that  are  identical  to  the 
manuals  used  the  fleet  squadrons.  With  this  new  system,  a  90%  revision 
reduction  time  is  estimated  for  the  subject  matter  experts  and  track 
managers.  The  costs  incurred  by  the  ISD,  graphics,  and  printing 
departments,  will  not  decline  with  a  reduction  in  the  number  of  workbooks 
or  revisions  because  nearly  all  of  their  costs  are  assigned  to  workbook 
revi£:ion.  Their  toted  costs  will  remain  the  same  because  ISD,  graphics,  and 
printing  are  now  staffed  at  a  minimum  functional  level  so  that  further 
reductions  are  not  likely.  The  cost  of  maintaining  workbook  currency  with 
fewer  workbooks  and  the  new  workbook  design  is  shown  in  Table  5.17. 
The  "new  system"  cost  also  includes  the  operating  budgets  from  Table  5.9 
as  before.  Tables  5.18  and  5.19  show  the  costs  of  workbook  and  lecture 
revisions  that  were  consolidated  in  Table  5.17. 
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TABLE  5.17.  THE  CHANGE  IN  CURRENCY  COST  WITH 
FEWER  WORKBOOKS  AND  LECTURES 


Measure 

VP-31 

VP-30 

Old  Currency  $ 

$968,790 

$125,750 

New  system 

528,320 

81,741 

Difference 

$440,450  (45%  less) 

$44,010  (35%  less) 

TABLE  5.18.  COSTS  FOR  MAJOR  AND  MINOR  REVISIONS, 
AND  ANNUAL  REVIEW  OF  WORKBOOKS* 


Position 


Cost  For 
Revisions 


MAJOR  REVISION 

Subject  Matter  Expert 

$11,670 

Track  Manager 

8,140 

ISD  Department 

84,750 

Graphics 

71,340 

Print 

48,000 

Sub  Total 

$223,900 

MINOR  REVISION 

Subject  Matter  Expert 

$3,140 

Track  Manager 

3,960 

ISD  Department 

72,190 

Graphics 

60,770 

Print 

70,080 

Sub  Total 

$210,140 

ANNUAL  REVIEW 

Subject  Matter  Expert 

$690 

Track  Manager 

590 

Sub  Total 

$1,280 

TOTAL 

$435,320 

*  Based  on  550  workbooks  and  90%  less  revision  time  for 
subject  matter  experts  and  track  managers. 
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TABLE  5.19.  COSTS  OF  LECTURE  REVISIONS 
BY  TRAINING  TRACK* 


Instructor 

Major  Revision 
Hours  Dollars 

Minor 

Hours 

Revision 

Dollars 

VP-31 

Pilot  (03) 

297 

$9,190 

102 

$3,160 

Naval  Flight  Off.  (03) 

451 

13,950 

155 

4,800 

Flight  Eng.  (E-6) 

887 

15,820 

304 

5,420 

Acoustic  (E-6) 

867 

15,460 

297 

5,300 

Non-Acoustic  (E-6) 

1116 

19,900 

383 

6,830 

Ordnanceman  (E-6) 

348 

6,210 

119 

2,130 

Inflight  Tech.  (E-6) 

75 

1,340 

25 

450 

TOTAL 

$53,220 

$18,260 

VP-30 

Pilot  (03) 

499 

$15,440 

171 

$5,290 

Naval  Flight  Off.  (03) 

736 

22,770 

253 

7,830 

Flight  Eng.  (E-6) 

887 

15,820 

304 

5,420 

Acoustic  (E-6) 

657 

11,720 

225 

4,020 

Non -Acoustic  (E-6) 

1140 

20,330 

391 

6,980 

Ordnanceman  (E-6) 

348 

6,210 

119 

2,130 

Inflight  Tech.  (E-6) 

75 

1,340 

25 

450 

TOTAL 

$60,860 

$20,880 

*  Based  on  a  35%  reduction  in  the  number  of  lectures  that  require 
revision  brought  about  by  reducing  the  number  of  aircraft  models  used 
in  the  fleet. 


The  Navy  is  faced  with  force  cutbacks  and  base  closures  which  could 
affect  the  number  of  students  in  the  Fleet  Replacement  Squadrons.  Table 
5.20  shows  the  annual  demand  for  students  under  two  different  scenarios. 
The  numbers  under  "fewer"  are  the  annual  number  of  students  that  would 
require  training  after  cutting  one  P-3  squadron  from  each  coast  and 
reducing  the  number  of  crews  per  squadron  to  nine.  The  numbers  under 
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"more"  are  the  number  of  students  that  would  require  training  if  the 
same  cuts  anticipated  in  the  first  scenario  were  made  and  if  aircrew 
training  was  consolidated  at  either  VP-31  or  VP-30. 


TABLE  5.20.  FLEET  REPLACEMENT  SQUADRON  STUDENT  LOADING 


Aircrew  Position 

Number  of  Students 

Status  Quo  Fewer  More 

Pilot 

120 

99 

198 

Naval  Flight  Officer 

80 

66 

132 

Flight  Engineer 

80 

66 

132 

Acoustic  Operator 

69 

57 

114 

Non-Acoustic  Operator 

35 

28 

56 

Ordnanceman 

40 

33 

66 

Inflight  Technician 

40 

33 

66 

The  variability  of  production  costs  over  a  range  of  student  loading, 
shown  in  Table  5.21,  was  estimated  using  the  number  of  students  displayed 
in  Table  5.20.  The  costs  of  the  instructor  time  spent  with  the  media  were 
assumed  to  stay  the  same  as  those  computed  in  Table  5.10.  Student  costs 
were  computed  in  the  same  manner  as  earlier.  The  costs  shown  in  Tables 
5.22  and  5.23  are  the  costs  that  were  used  to  derive  these  production 
costs.  The  production  cost,  which  was  defined  as  the  time  the  students 
and  instructors  spend  with  the  training  medium,  is  higher  for  VP-31  over 
all  ranges  of  students.  As  student  loading  increases,  the  costs  for  both 
methods  increases  at  the  same  rate. 
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TABLE  5.21.  THE  CHANGE  IN  PRODUCTION  COST  WITH 
CHANGES  IN  STUDENT  LOADING 


Measure 

Workbooks 

Lectures 

Status  Quo 

$2,929,700 

$2,567,040 

Fewer  Students 

2,465,800 

2,152,400 

More  Students 

4,619,000 

4,080,480 

TABLE  5.22.  COSTS  PER  COURSE  FOR  STUDENTS  TIME  WITH  MEDIA* 


Instructor 

vp- 

LC 

•31 

LECT 

vp- 

LC 

30 

LECT 

Pilot 

$71,200 

$13,350 

$55,410 

$28,040 

Naval  Flight  Off. 

30,620 

13,530 

33,820 

27,590 

Plight  Eng. 

8,600 

21,870 

10,740 

27,340 

Acoustic 

16,720 

15,320 

17,490 

14,520 

Non-Acoustic 

9,450 

9,690 

7,520 

12,370 

Ordnanceman 

70 

3,020 

90 

3,780 

Inflight  Tech. 

1,140 

770 

1,420 

970 

Sub  Totoii 

$137,800 

$77,550 

$126,490 

$114,610 

TOTALS  (LC+LECT) 

$215,350 

$241,100 

*  Based  on  11  squadrons  per  coast  and  nine  crews  per  squadron. 


If  the  workbook  training  method  had  worked  as  originally  designed, 
lectures  and  learning  center  instructors  would  not  have  been  required. 
The  learning  center  hours  shown  in  Table  5.24  under  VP-31  are  the 
learning  center  hours  prescribed  by  the  original  ISD  design.  As  an 
estimate  of  the  opportunity  costs  incurred  because  of  these  changes,  the 
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TABLE  5.23.  COSTS  PER  COURSE  FOR  STUDENTS  TIME  HITH  MEDIA* 


Ir  ■''tructor 

vp. 

LC 

-31 

LECT 

VP- 

LC 

30 

LECT 

Pilot 

$142,410 

$26,700 

$110,810 

$56,070 

Naval  Flight  Off. 

61,240 

27,060 

67,640 

55,180 

Flight  Eng. 

17,180 

43,740 

21,480 

54,670 

Acoustic 

33,430 

30,640 

34,970 

29,030 

Non-Acoustic 

18,900 

19,380 

15,030 

24,740 

Ordnanceman 

140 

6,040 

170 

7,550 

Inflight  Tech. 

2,270 

1,540 

2,840 

1,930 

Sub  Total 

$275,570 

$155,100 

$252,940 

$229,170 

TOTALS  (LC+LECT) 

$430,670 

$482,110 

*  Based  on  22  squadrons  and  nine  crews  per  squadron. 


cost  of  the  originally  designed  method  was  estimated  and  is  shown  in  Table 
5.25.  The  table  also  compares  the  present  currency  cost  of  workbooks 
only,  and  the  19fc0  Thode-Walker  estimate  [Ref.  17]  of  maintaining  workbook 
currency.  The  present  workbook  currency  cost  was  obtained  by 
subtracting  the  costs  required  to  maintain  lecture  currency  from  the 
currency  cost  in  Table  5.15.  The  developer  estimate  was  obtained  by 
applying  current  salaries  to  the  computations  used  in  the  Thode-Walker 
study,  and  adding  the  graphics  and  printing  department  costs  that  were 
not  iiicluded  in  their  study. 
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TABLE  5.24.  LEARNING  CENTER  AND  LECTURE  HOURS  PER  COURSE 

AT  VP-31  AND  VP-30* 


VP-31  VP-30 

Aircrew  Position  Learning  Cntr.  Lecture  Learning  Cntr.  Lecture 


Pilot  (01) 

Stadgnfr 

248 

Instt 

18 

0 

249  18 

126 

Naval  FUght  Off. (01) 

386 

27 

0 

228 

27 

186 

Flight  Eng.  (E-5) 

265 

16 

0 

88 

16 

224 

Acoustic  (E-4) 

229 

65 

0 

200 

65 

166 

Non-Acoustic  (E-4) 

154 

60 

0 

175 

60 

288 

Ordnanceman  (E-3) 

40 

0 

0 

2 

0 

88 

InfUght  Tech.  (E-4) 

28 

20 

0 

28 

20 

19 

*  The  student  hours  listed  under  VP-31  were  determined  during  the  ISD 
process  used  to  design  the  original  training  program. 


TABLE  5.25.  PRESENT  ANNUAL  COSTS  COMPARED  WITH  COSTS 


ESTIMATED  AT 

INCEPTION 

Measure 

VP-31  Now 

Developer  Est. 

Production  Process 

$2,929,700 

$1,736,860 

Currency 

858,830 

582,990 

TOTAL 

$3,788,530 

$2,319,850* 

*  Includes  graphics  and  printing  department  costs  which  were  not 
included  in  the  Thode-Walker  study. 
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VI.  SUMMARY  AND  CONCLUSIONS 


This  study  compares  two  methods  of  instruction,  workbooks  and 
lectures,  as  presently  used  in  P-3  Fleet  Replacement  Squadrons,  to 
determine  which  method  of  instruction  is  more  efficient.  The  study 
compares  the  training  program  at  VP-31  that  uses  workbooks  and  lectures 
in  a  nearly  one  to  one  ratio,  with  the  training  program  at  VP-30  that 
supplements  lectures  with  only  a  few  workbooks.  It  is  again  important  to 
emphasize  that  this  study  does  not  look  at  the  training  costs  incurred  by 
such  things  as  aircraft,  maintenance,  facilities,  and  administration  costs. 

The  lecture  training  method  at  VP-30  is  much  more  efficient  than  the 
workbook-lecture  method  at  VP-31  based  on  the  operating  costs  presented 
in  Table  5.15,  assuming  equal  effectiveness  of  both  training  systems.  As 
seen  in  Table  5.15,  the  total  annual  opportunity  costs  associated  with 
teaching  the  majority  of  the  theoretical  information  through  lectures,  are 
70%  of  the  costs  of  the  workbook  and  lecture  method  used  at  VP-31. 

The  higher  costs  of  the  workbook -lecture  method  are  due  to  higher 
student  production  and  revision  costs.  Production  costs  at  VP-31  are 
higher  primarily  because  of  the  additional  time  instructors  are  assigned 
to  the  learning  centers;  a  requirement  which  was  unanticipated  when 
selecting  the  workbook  medium.  Revision  costs  are  higher  at  VP-31 
because  of  the  extensive  effort  required  to  keep  the  workbook  medium 
current  as  new  software,  equipment,  and  tactics,  enter  the  fleet. 
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Production  costs  were  estimated  for  a  range  of  student  loadings  that 
could  result  from  two  possible  scenarios;  reduced  student  loading  from 
lowered  fleet  demand  because  of  fleet  squadron  reductions,  and  increased 
student  loading  caused  by  closing  one  of  the  Fleet  Replacement  Squadrons. 
In  both  cases,  the  lecture  method  was  still  found  to  be  the  most  efficient 
method  of  instruction. 

The  cost  of  maintaining  media  currency  may  decrease  for  both 
training  commands  in  the  future.  Substantially  fewer  revisions  to 
workbooks  and  lectures  are  expected  as  a  result  of  a  reduced  number  of 
P-3  aircraft  models,  and  as  a  result  of  a  new  workbook  design  that 
removes  the  most  frequently  changing  information.  VP-31's  training 
program  will  benefit  the  most  from  fewer  revisions  but  the  currency  cost 
at  VP-31  wHl  remain  considerably  higher  than  VP-30.  The  higher  currency 
cost  under  this  scenario  is  due  primarily  to  the  manpower  requirements 
of  the  workbook  revision  process. 

The  workbook -lecture  method  used  at  VP-31  does  have  a  lower 
portability  cost.  Even  though  the  immediate  need  for  portability  appears 
to  be  declining  because  the  changes  to  fleet  squadron  aircraft  should  be 
completed  by  1992,  new  aircraft  models  will  follow,  and  the  portability  of 
a  medium  will  eiiways  be  a  consideration. 

Certain  qualitative  attributes  should  also  be  considered  when 
evaluating  the  efficiency  and  effectiveness  of  training  programs.  The 
workbook  method  appears  to  have  an  advantage  in  its  ability  to  contend 
with  instructor  absences  and  its  ability  to  maintain  instructional  material 
standardization  although  it  can  be  hindered  by  production  worker  absence. 
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If  the  workbook  method  of  instruction  had  been  maintained  exactly  as 
designed  by  the  Instructional  Systems  Development  (ISD)  program,  t'.e 
workbook  method  would  now  cost  much  less  than  VP-31's  present  program, 
and  only  87%  of  the  lecture  method  now  used  at  VP-30.  At  VP-31,  the 
production  process  costs  are  much  higher  than  anticipated  because  of  the 
necessity  of  adding  a  learning  center  instructor,  and  the  requirement  to 
add  numerous  lectures  to  compensate  for  inadequate  workbooks.  The 
currency  cost  is  higher  than  anticipated  as  a  result  of  underestimating 
the  revision  effort.  Revisions  were  required  far  more  often  and  required 
the  efforts  of  many  more  workers  than  planned  for. 

The  cost  differences  between  VP-31's  present  costs  and  the  "what 
should  have  been  costs"  based  on  the  origined  design  indicate  that  the  ISD 
system  that  chose  the  workbook  method  for  the  primary  means  of 
instruction  is  flawed  because  the  workbooks  were  not  entirely  satisfactory 
for  instruction.  The  ISD  program  failed  to  allow  for  the  boredom 
associated  with  studying  workbooks  for  hours  on  end  without  student- 
iristructor  interaction.  Implementation  of  the  workbook  system  failed  to 
take  into  account  the  ramifications  of  applying  the  rapidly  changing 
information  to  a  fixed  and  difficult  to  change  medium. 

The  ISD  system  could  also  have  problems  because  it  is  not  well 
adapted  to  the  realities  of  the  military.  The  ISD  system  uses  a  few 
experts  to  organize  and  lead  a  team  of  subject  matter  experts  to  develop 
a  training  program,  but  once  the  program  is  developed  it  is  used  in  a 
command  composed  mostly  of  transient  military  people  who  are  given  the 
responsibility  to  maintain  the  curriculum  as  it  was  designed  but  who  are 
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unfamiliar  with  ISD.  Within  a  few  years  of  introducing  the  P-3  ISD 
designed  training  program,  few  if  any  of  the  major  changes  made  had  been 
validated.  Even  more  problematic  is  that  during  those  early  years  fleet 
reports  of  poorly  tredned  students  motivated  many  of  the  changes  which 
did  occur.  This  raises  the  question  of  whether  or  not  the  original  ISD 
program  was  more  effective  than  the  program  it  replaced.  The  training 
programs  now  in  place  at  VP-31  and  VP-30  are  meeting  the  fleet  needs 
through  training  programs  that  have  evolved  considerably  from  the 
original  ISD  design,  and  those  changes  were  made  by  instructors  doing 
what  they  thought  was  best  without  heavy  development,  implementation, 
and  validation  costs. 

Because  millions  of  dollars  are  spent  by  the  Navy  on  ISD  programs, 
whether  or  not  training  effectiveness  is  increased  as  a  result  of  using  the 
ISD  process  to  design  a  curriculum  would  be  an  interesting  future 
research  topic.  Perhaps  it  is  better  to  save  the  millions  of  dollars  and 
instead  rely  on  the  intuition  and  expertise  of  the  managers  and  instructors 
in  the  training  program  to  make  changes  as  needed. 

When  selecting  training  media  alternatives,  a  cost-benefit  ancQysis  that 
compares  the  opportunity  costs  associated  with  the  alternatives  should  be 
used  as  the  measure  for  comparison.  To  be  accepted,  a  new  method  of 
instruction  should  compare  favorably  with  conventional  instruction.  A 
cost-benefit  analysis  that  evaluates  a  new  technology  should  be  careful  not 
to  underestimate  the  effort  required  to  maintain  the  currency  of  the 

f  material  presented  by  the  technology  and  not  underestimate  the  manpower 

demands  of  using  the  new  technology. 

> 
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